Modern Browser Performance Testing Master Class with Neotys PAC

Welcome to the Test Guild Performance and.

Site Reliability Podcast where we all get.

Together to learn more about performance testing.

With your host, Joe Calantonio.

Hey, it's Joe. And welcome to another episode of the Test Guild Performance and Site Reliability Podcast. This is going to be a recording I created for a meetup I actually was a guest on for Neotis Meetup run by Henrix Rex. It's actually a killer lineup that I was a part of. I spoke for maybe 30 seconds, but I was lucky enough to be on the call to sit back and listen to the expertise on this panel that included Ian Scott Stevens, Arthur Loomis, Bruno Du, Leandro Melendez, Alexander Podelco, Mark Tomlinson, David Johns. So a lot of great thought leaders in this area all about browser rendering and DOM loading. Performance was the topic. So it was about a month ago. It was so good I wanted to share with you as well. So sit back. It's a little longer than normal, but like I said, it's like a masterclass of what Neotus calls their PACK or their Performance Advisory Council. That's why they call it pack. So you don't want to miss this episode. Check it out. This episode is brought to you by SmartBear. Listen, load testing is tough. Investing in the right tools to automate tests, identify bottlenecks and resolve issues quickly could save your organization time and money. SmartBear offers a suite of performance tools like Load Ninja, which is a SaaS UI load testing tool, and LoadUiPro, an API load testing tool to help teams get full visibility into UI and API performance so you can release and recover faster than ever. Give it a shot. It's free and easy to try. Head on over to smartbear.com solutions performancetesting to learn more.

I think that Joe asking oh no, that Sartre ask. So what is the performance measurement? What are the measurements that we can make? Is it the response time of the page? Is it the first byte's appearance? Is it interaction moment? We see, as our friend of Dynatraz show us, we see there are many indicators that Dynatraz proposed. They don't care about which is the best indicators. They just say okay, we have many indicators. We have an indicator of page download time to first byte and after indicators that the DOM is complete enough and the full interaction is possible in the page. All these indicators are different In Dynatraz they propose all the indicator to be sure that you pick up your indicator, you do what you want so you can choose which indicator is yours. But that means that Actually, we don't know which is a good indicator in this new problematic.

Single failed application. I don't know what is the best indicator, but my reference is always the customer. When they tell me this page is longer, I always ask them explain me what does it long for you? Is this long because it takes time for appearance, or it takes time to download, or it takes times when you click to have something interact with you. And.

It depends of the people.

When it's IT people, they tell me it's long because in the network it's long. When it's people from the business, they tell me it's long because after several seconds we want to move from this page, it's too long to come. So it's a question of perception. That's the difficulty of our business today. And that's why user experience meetup is complicated. Because this performance indicator, I don't know which is a good one and it could be different from a a company and to another it could be different from the inside application and from the E business application. So of course the good point of view is to give all the indicator and to compare all the indicator like they do in Dynatrace. Do what you want with your indicator. You can make the graphs that you want and you can compare after what you want. But let me tell you something about pure performance. In the old way, before the single page application, we spend many times to optimize the server, to design a page, to design a full page HTML page with some good CSS and some good GPA. A page made around 400 milliseconds to be designed and broadcast in a navigator, 400 milliseconds. It's very quick when you design a complex page.

A good design page. So. So I spend many times with my clients to design, to optimize server able to deliver between 10 and 12 page per second per CPU indicator, 10 page per second per CPU. This is an indicator that I use to talk to my clients about their architecture and the ability to deliver a very quick page. Now we have another problem. They optimize all the server. They spend millions and millions of euro to optimize the server, hybrid server, Oracle server, Java server, to optimize and to design, to create page. And now the problem is the time is spent in the browser and it doesn't depend about.

Their server, it depends about the CPU of the user, the CPU of the Navigator, the memory of Chrome, of Internet Explorer. It depends on something. They don't have the end on it. In companies they can have the hand on telling everybody is using Internet Explorer 8.11 etc. But in E business it's not possible. So they are working to spend a lot of money in CDN in order to have cash everywhere in order to to manage this problem of response time with Many requests many JavaScript many calling of CSS when the Xashare is calling a new request. So now the money is not spent in the server in optimization of server it spend of designing page. That's what you said at the beginning. And so the problematic is not coming from us how to design page is coming from which framework is the best one. And this question is not alive. I don't see this question everywhere. I don't see this question anywhere. So this could be the good question which is a good framework to design very quick page and does this framework give you indicator when you design a page does it give you indicator to make quick page?

Thanks Bruno for that. I think there is a From what you mentioned, there is one thing which is important is the and it's been mentioned by Jan and the others. From the moment you start to have to be able to click somewhere or to feel that it's been loaded, the customer seems happy. I mean if you look at Amazon you can have the page you start to browse and click on everywhere but in fact in real life the rest is still loading in the back end. And as a customer we don't complain, we are happy about it. And this is a write direct API. When is the customer is able to officially interact with it. And I think there is a big challenge. I think there is the traditional webpage with tons of JavaScript and there is this concept that's been mentioned several times, the single page apps or concept of lazy loading.

And where every time you do something there is no loading anymore. It's just some JavaScript that changes structure of the page. Lively. So this concept of when should I interact or when can I touch the screen is very difficult to measure and I have no clear answer. I mean there has been by the way someone in the last virtual pack we had a great presentation about it. So how can we time and measure on the browser? So at the moment there is no clear answer. I mean I think it's instrumentation and Nihandro just raised just the hands. Maybe you have something about it. Do you have any approach.

Outside of just instrumenting your JavaScript code or your web page to measure where you're spending the time and collect that? Do you have another smarter approach?

Well.

It'S funny because as you mentioned, there are multiple ways as everything with performance testing. A good answer generally is it depends on what you're trying to do, what do you need, what are the goals and what is the nature of the application. There were some comments that as an example when Scott was mentioning the experience of arriving at a customer that you need to probably low test automate something. And I do agree the first step is like well, let me just see the page render pop up the developer tools. Let's see what can I observe right there right away and start doing some assessments depending. And this is one where I would say you require to have some sort of experience working with this type of things to understand.

Like the girl in the red dress in the matrix. It's just code flowing but you still see it right there when you get enough experience.

And you don't even need the rendering tool happening. This is something that I couldn't say. It's one way or the only way that you can gather this. It's based off experience different situations. As Arthur was asking, what is more important when you receive all the code or when the page finished rendering? Again, it depends on what do you need. Some of these metrics that you were asking Hendrik will be very different. As an example, let's say what is the problem?

Some design teams or functional teams that I saw someone asking if this shouldn't be catched on the functional phase layer of the testing process. Some of these people do not even have mindset experience or the eyes to look at that matrix code and differentiate what is happening when.

I'll give you a quick example, a couple of quick examples humongous JavaScript files that you say, well, you think it's a very smart way to do. Why I don't Download all the JavaScript that will be needed. Even as the user will only click one thing and get away. You're forcing them to Download a humongous JavaScript file when it's not needed. Probably even worse because the web developer again probably was a designer or someone that was not that aware of that. The expertise area of this person is more on making things look attractive. Probably they put that humongous JavaScript code in the server that has a web application as well. Do not use a cdn, do not distribute this.

This is an example that I have seen lately on many organizations. Just because this front end allows you to just bring a lot of information and work with that kind of offline, they are like okay, the time that the user arrives at the lands at the page, I'm going to download all the freaking database of records that we have just in case they have to work. They're like, there's no problem. Well, right now probably there is not problem, but let's wait on two scenarios. When 100 person people try to access the page at the same time and everyone tries to download that huge amount of data, that's going to be a problem. And down the road, when your data grows large enough and you have billions and billions of records, as Carl Sagan would say, you would have a big problem there where.

Just one user tries to download all the applications. So again, thinking things through CPU cycles, how much are you forcing to not only the CPU to work the network to communicate, how much ram, how much stuff are you storing locally? It's a complete full beast that you need to pay attention to. And the metrics and things that you can put to detect this type of things, again, are extensive, very varied. To be able to do a one ring rules them all solution.

So.

Scott, you raise your hand.

Yeah.

A comment was made, I can't remember by who, and I apologize for that.

About.

Issues with performance on the back end because a user's machine was under spec'd on the front end and the CPU was grinding through and basically session states and things like that were being held open for a very, very long time because the local machine couldn't process what it needed to process fast enough. And therefore session, et cetera, et cetera, et cetera, is being held on the server.

It's almost in some ways going back to like the bad old days where if you had a whole lot of users, for instance, who were connecting over 33, six modems, those were the guys that would kill your system as opposed to the ones that were coming in over broadband because again, that connection had to be held for so much longer to get the data backwards and forwards and things like that through the pipe. It's almost like a case in some ways of what lessons can we learn from the past that can we apply today? And it's like if the single page app is going to execute locally on the user's machine in like one second for it to go through and do all of its parsing and stuff like that at the protocol layer, can we increase the concurrency across that? So like we do in any other load test, it's like my user might be in for five minutes, but for this I'm going to vary the session state by say 25% either way and start really, really increasing that currency and see where things start. Falling over.

To Leandro's point. I've worked on a system that did exactly what he was saying about six years ago. And it was an enterprise system. And we couldn't figure out why until basically we put Fiddler on and just sat there and watched this thing while we were doing nothing. And we could start seeing all of these calls coming out and figure out that this is the one that we need to simulate. And we trace that call all the way back to the mainframe. And sure enough, it was bringing back everybody's data back to the front end, selecting the record that needed throw everything else away and then repeat the call. So it would get your first name, then it would get your second name, then it would get your address, et cetera, et cetera, et cetera. Each one of these was a full call down to the mainframe. It can be done without a tool or without a browser, but it requires that extra little bit of expertise, I think, and a little bit of sheer bloody mindedness to sit there and try and figure it out.

It reminds me when you had to deal with the modern web apps where and you had to script or build at least a load testing script. People were recording and say, I'm going to record and playback. But in real life you should not do that. You should do what exactly you described. Look at what happens on Fiddler when you do nothing. Maybe they asked.

Generated by JavaScript and you have maybe wants to build a representative protocol based script that reflect what you just described. I totally agree with you. So David, you just raised the hand.

Yeah, no, because what Scott was talking about is just, it's so compelling and interesting and I've run into it so many times now, especially with people with existing enterprise applications. I put a note of it into the chat and that is I've seen lots of insurance companies running software in IE compatibility mode. When they do that, essentially all of that parallelism that modern browsers allow for in terms of supporting parallel connections goes away. So you can only have two connections per domain. And in that case you get this situation where the client is introducing a instant bottleneck that gets felt by everyone that's trying to connect to that server because that client is now consuming a whole bunch of connection ports and things like that. It's surprising how often this actually comes up. And for the most part people become absolutely mystified when you start talking about the details in terms of how browsers actually behave when it comes to handling things like parallel connections. So that is what Scott brought up was, I thought very Very interesting. And I can tell you for a fact I have seen this in the real world and continue see it today.

And moreover more we're going to see more and more for sure. Alexander, you tried to raise your hands, but your hands are like invisible.

Yeah, background is tricky stuff. I want to change topic a little and bring it to tools. So it looks like we have at least four silos here. People here probably more familiar with traditional load testing tools. Then we have functional testing tools which now we get more and more get involved with promise that easier script creation. But there is a lot of problems here because usually they were not designed with measuring performance in mind and actually no information.

What they support. Because for load testing tools you usually know which protocol they support. For GUI tools you usually need to try to figure out what is it working or not. If there is some components that may be not playing HTTP there, then we have these tools which allow to see what happens client side that communication like webpage test.

Site, speed IO which create tools a lot information if you have single page or few pages. But as Henrik mentioned, if you have the scenario when you need to log in and do something and have a lot of calls going on, it is big change how you can use it. And actually they also not catching everything. For example JavaScript basically left behind. So you need these browser development tools to see that. So at least four different silos. And the question is how to get them all together work to work. And for example, I work with applications that have all these problems. Single page application, a lot of not HTTP elements there like working with Word documents or PowerPoint documents in browser. And of course sophisticated scenario when you go from one point to another. And that is definitely a challenge. And it become big challenge now. Especially when you work in development and need to get some kind of continuous performance testing server is not enough anymore. You need to get everything to catch all kind of regressions.

Yeah, I agree. So we usually try to recommend combining just for the sake of the cost of the hardware required for load testing to do protocol level testing because it's going to be cheaper and then add on top of that few browser based sort of technologies to get the feeling, to get the perception to get the feedback on.

How fast the page were marked. You here and you've been presenting stuff about web performance our virtual package. I'm pretty sure that you have tons of things to tell about that topic today.

Let me share one thing that might be interesting different than all of the excellent technical analysis. You guys are right on. Absolutely. I learn from you guys, every day, some of the things that all the different tools can bring together. But an interesting story from my perspective about 15 years ago, witnessing our granddaddy loadrunner do its work at the protocol level and making great market strides supporting all sorts of new protocols. The future of the web. And I had worked on the product at a time when Web 2.0 and AJAX frameworks and all this stuff was getting really interesting. Of course, you guys might remember that because of the market dominance of that product, that we had basically told a generation of load testers, everything you need is in loadrunner. That's all you need. All you need is loadrunner. And to be a performance tester or performance engineer, performance anything. You're the cream of the crop. You're the certified this, that and the other thing. At the same time, I had taken this break from life. I'd met Andy Grabner working for Segway and then working for Dynatrace, and I'd worked with profilers and I'd done all sorts of kernel profiling and wow, there's a whole lot more to performance than just a load testing tool. And I was a firm believer in integrations. And that was one of the things that we, when I did get a chance to work with the loadrunner product, I had to fight for with many different people. A good argument, debate is good, but really have to make a business argument for the fact that the definition of a performance problem was much bigger than just load. To Alex's point, there are other silos of what is a bottleneck? Where is a bottleneck happening? How do I find that bottleneck to David's point? And how do I make it easy to solve it and actually get people on a good path for how do I think, like a performance problem can happen anywhere in the architecture? And I gotta maybe put together six or seven different tools to try to do that in an efficient way across multiple teams. So when we came out with the whole LoadRunner solution for, you know, running Firefox or running a DOM, running the JavaScript engine, it was another approach. We looked at some really interesting solutions and. But still we had this, I don't want to call it an army, but we had this huge community of people who were load testers, 90% of them. So if you came to them and said, hey, I think something's slow on the front end over here, like Bruno's conversation, it was like, oh, the answer was, somebody else does that.

In the chat. The developer should take care of that. Or, gee, 10 years have gone by Steve Souders. Hey, isn't he a developer? That's not my community. That's not. It didn't show up at the HP Universe Mercury Conference, whatever no one was talking about. Wow, why weren't we talking about that? So I think there's something about the shakeup we've seen and.

Henrik knows this really, really well in terms of how the different load tools have come to expand.

And embrace or be inclusive of all the different places that you can find a performance problem.

To some extent I'm responsible for saying, yeah, I'm a load tester, but it's not enough. We have to change. For me, that was 10 years ago and started going off to do other stuff with Shunra at the network level, working in the other applications that were doing cloud based load testing and finding other infrastructure issues in the cloud, which led to the whole conversations about cdn. Then within the last couple of years, you're right, Henrik, it's been like, how do we do a better job with tools like Fiddler and doing a lot of work with Dynatrace to just tune into how are we getting that full picture of this stuff? There's something to be said for I'm a load tester. Well, that's only one silo of the three or four or five silos. And your thinking should be one silo. The whole big thing. You should be an evangelist or enthusiast for performance. Wherever these problems may happen and whatever tool you might use, bring them together, put them together just like David said. So yeah, definitely.

And I'm happy that you bring up the network emulation because there are still so many people who not even consider putting a network constraints when they do load testing. They are pure load testers. So do scripting. They fire the load and say, a network is not my problem. I'm going to fire and whatever it is, I'm going to measure the performance. And this is just an example. And you're totally true people. There are still people say I'm doing performance testing, but in fact you're doing just load testing.

Right.

And I think we have to open up and say, hey, there are plenty of various things that you need to consider when you deal with performance, not only load testing. So I'm pretty, you know, I'll just.

Give one shout out. Many years ago I did, I did a workshop, like a half day workshop called slow motion performance and it was exactly doing front end performance. You know, just the rules, like why slow rules? And you would run like just any public website, anyone could bring their website and we would check the rules when you're on full speed network and then we would run it through either Squid or Charles Proxy or whatever. You want to slow down the network and run the exact same thing, just client side rules. Actually later I think David, Andy and I did it with the AJAX stuff from Dynatrace. We did the same analysis. And of course suddenly people start seeing the order and sequence of stuff rendering and then CSS doing its work. And it's all based on when the resources are downloaded. And you see this really suddenly everything's in slow motion. Be like, oh, what was that? What was that? And why isn't the first thing that renders the give me money button in E Commerce do the thing I want you to do button should show up first, then put the pictures and the colors and all the other stuff. One of the biggest ones was watching Amazon at the time Amazon.com render because they had done this amalgamate webpage with all this asynchronous loading. Fat people were, oh my God, I've never seen like the actual slow sequence. But you can find all these interesting other bugs. You can see stuff that loads. You know, that's like, wow, that's weird. That's actually, oh, look, there's an object that we can't even see when it fully renders. And you can go like functional exploratory testing. You can tear the web page apart because there's these hidden things in there. Slow motion performance is a totally fun thing to do.

Yeah.

I like Mark's remark about what comes first on the page because that's really what is most important to the end user is to make sure that what he actually came for on that page is actually shown as soon as possible. And I think it's not just that it's shown as soon as possible, but also when it's there, it actually stays on that page. Right. And it's not being then when you start to read the blog message that is suddenly pushed down because there's advertisements coming up front. So you can say, okay, my first meaningful paint is there really soon. So I start to read, but then I lose it because there is another ad or something else that moves on the page.

So.

And I think there are other aspects as well. There are aspects to perceived performance which have to do with if you have an animation. So you click on something and a new page comes up or a button appears or whatever that is. Animation is done in a certain frame rate which looks that it's fluent, like It's a fluid motion. You click on it and it open up like that and not stutter like that because it appears slow again. So I think there are many, many aspects to.

Perceive performance which are very interesting and they're much wider than just a response time measurement or something that we could put a second on. And many of them have some sort of a psychological aspect to them. Right. Well, how does the page feel? So suddenly psychology, or at least some sort of awareness of how that page will create a kind of feeling for the end user suddenly becomes important. Now that for a lot of our techies is a very difficult thing to grow.

So.

How do we now suddenly define something which seems to be so soft? So that's one of the points I wanted to mention. Another point I wanted to mention is that we talked about browser based load testing. I think that's really what we're going for. It's not just that we can reuse the Selenium scripts or the scripts that our friends from the, from the GUI tests are creating, but also it really will make our load testing scripts a lot easier, I think, because you only have to deal with what is on the screen and not all these different AJAX calls which are going behind, not all the parallelism. So now the question is how would you deal with that? And that's why I think here is where the cloud comes in and where the hardware which is getting cheaper like by the month, will really help us. So maybe not this year, but next year I think we will be able to perform browser based load tests, simulating hundreds, maybe thousands of browsers at the same time and really, really do what the end user is doing and really, really see and measure with tools like Dynatrace for instance, or other tools to really, really measure what the end user is seeing. And I think that is really the end goal of, of what performance management is about.

But do you need thousands and millions of browser to achieve that?

It's a good point. You could actually say, well, I want to simulate a million user coming to my website, but they're not all coming at the same time.

Right.

So what you could do is you can take say a couple of hundred of browsers, open them up on pods or on Kubernetes or any kind of cloud solution that you, that you have, and then make them run simultaneously. Right. So that the number of concurrent browsers is still a lot less than what you would have in reality, but you will still get the load on the server that you would normally get.

Yeah, but I think that you mentioned the feeling and I love the story that Vasily came up the last pack. I don't remember the exact story about the airport. I don't remember which airport he was referring. I'm trying to search in Google but I didn't find it. The story is that the passenger was complaining because every time when they were getting out from the airport on that specific airport, which one, I don't remember, they had to basically walk 12 minutes and then wait for the luggage for 10 or almost 50 minutes. So people will say, hey, it's unacceptable in this airport. We're waiting forever.

I'm complaining about it. So they make some study, say how can we change the perceptions? So instead of coming through terminal A, which was quite close to luggage, they were changed the arrival to terminal B. So now people were walking during 50 minutes, then going down to the stairs and waiting almost 30 seconds and get the luggage and people will say, wow, the service is amazing that airport, because it comes very, very fast. So everything is about perceptions. So I don't think that simulating thousands of browsers require.

Yeah, I just wanted to add to part to the comment. I wanted to get back to something that Arthur had talked about earlier on in the conversation that is trying to pick functionally what the right metric is. I just wanted to introduce everyone who's new to this just to have a view as to if you're going into the Chrome developer tools and doing a page load and you actually do the performance analysis, exactly what you get treated to. And I think one, this is an absolutely fabulous, fascinating view. But that being said, there's just so much detail in it for even a performance developer to try to understand where should they be applying and looking at.

Their efforts to improve performance. Because this is just literally loading an individual sample page that we had showed before. So trying to define and figure out what those metrics are that are going to be most important to you really does become an exceptionally important part of the conversation. You really do have to be comfortable with the metrics that you're going to get your reporting back on. And that's, you know, make sure that you're picking things and picking metrics. I know somebody had mentioned earlier.

I think Arthur, you had mentioned the first contentful paint as being a potential option. Here at Dynatrace we really gravitate towards meaningful paint. Yeah, the visually complete. But that being said, we know that there are people in the industry that are interested in some of these other metrics and so we're actually just sort of sneak Peek, sneak preview, let people know where we're heading. We're actually adding things like the first contentful paint, first meaningful paint, first time to meaningful interaction. So we're adding those metrics into Dynamics Trace as well, just to give people those options of being able to, you know, choose a metric that, you know, they feel comfortable with.

So before we just say goodbye and maybe we will probably organize another back meetup, I'd like to make the conclusion. So I will ask first Joe who his final thoughts, the final recommendations on the what the discussion we had just today.

So my final recommendations is low tech. I think Bruno said it best is that understand your users. I know it sounds obvious, but most people don't listen to the users. I just work for a hospital with radiologists and we used to not speak to them and not know they were saying something was slow because they weren't able to do their job fast enough in a particular area of the application that we never focused on. So looking at all the graphics, all the metrics in the world would never have told us that. So I mean, sounds stupid but you know, talk to your users, find out what performance means to them and then focus on that.

Yann?

Yeah, I agree, Joe. It's, it's at the end of the day, yeah, performance is a perception, percept, user perception thing. It varies across, you know, what, what the application is being used for, whether you're buying stuff, your radiologist, you're in some other, you know, and so good performance means different things, different people. But essentially it's the ability for them to use the application to, you know, so they don't get frustrated, they can buy something, they can do the job and that's key. At the end of the day that's, that's, that has to be, you know, the deliberate contesting that that important functionality is available and performant at load. It's an old message. And how do you achieve that? Well, it's always going to be some sort of hybrid test solution, protocol level or headless. Don't quite like that term actually it sounds a bit brutal, but protocol level plus plus functional testing combined effectively functional testing combined in the same sort of test. And why not bring back good old crowd testing? Maybe there's a case for a bunch of people sitting in a room with different browsers, different devices. We're running performance tests, we're doing some monitoring, but at the same time there's some real people doing stuff. I think it's a nice mix of.

Automation and reality. Anyway, that's my Thought it really is horses with horses. But focusing what in the browser is important to the end user? What do they need to click on? They will carry on to the next step. Yeah, the pictures aren't necessarily that important. It's the ability to interact, not be blocked, I think not be frustrated.

And that at peak times.

That's my thoughts.

Arthur.

Yeah, something I'd like to tell all the protocol based load testers who are watching this or listening to this and I think it's really time for all the load testers to also look at functional tools which they might think might be inferior. It's just gui, right? So get yourself a dummy course on how to use Selenium. Record your first Selenium script and let's try it out on different try it on different sites. And another tip I'd like to give everybody if you ever have problems convincing a manager at your company that that you should have some performance improvements and take that manager, show him his own side in developer tools, make the network just a little bit slower, let's say 3G or so and then load the page again and see if he still thinks he doesn't have a performance problem.

Good point, good point. Arthur Leandro Senor Performo.

Well, as those.

Things.

Very important.

Start paying attention to the web layer ui web protocol. I do agree a lot with Arthur that all these not especially but performance testers that are used to just automating protocol processes and slamming applications with them need to start thinking a little bit. I don't think it's out of the box anymore, but start expanding the reach and thinking of other elements that are starting to be really important. And that from what Mark was mentioning earlier brought to my attention. Think through how these web pages are being designed. How do you put the elements? Performance testing has to do a lot with the experience of the user. How much are they satisfied and how are you designing these things to the benefit of the user for them to be happy and enjoy and have a flowing fast and seamlessly transparent application where instead of putting CSS in the wrong place, huge JavaScripts huge. And what Mark got me thinking, huge images or videos in your webpage where probably you have a catalog full of ultra 4K images for each one of the products. Think. Start to think things through. And where am I going to put things when I'm going to crop them a little bit so that the page renders faster, that I'm not killing the cpu, the ram, the network of the application and that the information is flowing in the most effective way where again and to what Arthur was saying, introduce to your mindset some of the functional testing tools. What can you leverage from other UAT crowd testing people that are already running through some of those processes? Hey, why don't you just open the browser, tell them hey, could you just easy tell me if you feel it's low. That's a big, big advantage to the web protocol where probably there's no load happening and you can start detecting some of these problems. And it's a big complaint that I usually have when I get to pre exam, hey, you need to load test this and manually as Scott mentioned, I get to a customer, the thing is slow, I can just quickly open and say here you have a problem here, here, here, here. And wonder why no one ever mentioned this, why no one in the team, the Agile team, noticed it, mentioned it, reported it. Why is it not something to call your task completed your little post it and say I'm done. To say hey, it goes fast, it's seamless, I'm not annoyed by it and well, I'm extending too much just to resume. Pay attention to web expand your horizons as a performance tester and pay attention to the other levels, tiers, network slowing content thinking through a table that queries the request one row at a time. Think of those assign matters and yeah, I see some laughing. Everyone has experienced some of those things. Big, big performance scene to commit. Be careful with all of those.

Thanks Bruno.

Yeah, just to tell you that it's going to be very short because if the perse is today a question of feeling and so this is not anymore a question of technical. So it's not a question for us who are technical guys. This is not our job anymore. So I think that today we are dinosaur guys. That's probably the end of the job for us. Now we are dinosaur because we make all the job to optimize all this web server, all this architecture, front, back, et cetera. And now the question is not anymore our business. Maybe it's a question of perception. So let's work with social people who are able to explain to the customer or to the company how to spend 1, 2 or 3 seconds towards the information the website.

Thank you, Scott.

I think the interesting thing from my point of view, this isn't the case of either or. You don't do this or do browser testing. I think it's a case of both and I think it has to be both because any metrics you get from the front end are always going to be slightly fuzzy as to whether you're getting first meaningful paint or first interaction time, you're never going to be able to account for everybody's browser configuration and things like that. So you're looking for an inflection point almost as where does the biggest bottleneck occur at any point under low load, is it on the browser because something funny is configured under high load, is it on your back end infrastructure? I think to get that understanding you need to know what the components on the front end are doing and how they are interacting with the back end and you can simulate those at a protocol layer. If you need to have something very, very, very targeted, then you can do that at the protocol layer and you can have the browser sitting on top with less users, one user, selenium script, whatever to give you that delta and views.

I think too, while the browser stuff is probably the way of the future, always, always remember that there is an architecture sitting at the bottom of that that your users are communicating with. And part of your role is to make sure that that architecture is scalable, it has enough capacity to do what it does. Because if that falls over, it doesn't matter how efficient you make the front end, it's game over at that stage.

Alexander, I just want to mention that reason why we have silos with tools is because we have silos with groups of professionals. Every group was very specialized actually surprising because performance depends on everything. So it is not.

One small area where you need to look. It is a whole thing end to end and it worked for some time due to.

Level of technology, but looks like it is not working anymore. So definitely if we want to not to be dinosaurs, we need to get in all.

Performance end to end.

And whatever gaps we have between groups we need somehow fill them because that looks like the requirements now.

Thanks Alexander.

David?

Yeah, no, I would summarize things into two categories. One, I would speak to the front end developers and I would say to you this, just because you can do something doesn't mean that you should do something. And I'll just let that one sit and it's. I think most front end developers will get what I'm talking about. But for the, for the folks that are really involved in the load testing side of things, really my recommendation is when you're doing your load tests, you know, ultimately everyone's been giving great advice today. This has been, I think a fascinating subject, but make sure that you're always testing from an outside in perspective. When you can, you want to test the entirety of that application delivery chain because as so many of us has pointed out there are potential bottlenecks that you may have missed if you've only been doing your testing internally.

Test that entire application delivery chain. Make sure you have visibility into as many of the different components in that application delivery chain as you can. And that would be one of the things that I would sort of recommend.

Thanks again Henriks for allowing me to be one of the guests on your pack meetups. Really appreciate all you do for the performance community. For links to everything of value we covered in this episode, head on over to testguild.com P38. And while you're there, make sure to click on the Try Them Bolt Today link under the Exclusive Sponsor section to learn about Smart Bear's two awesome performance testing solutions solutions. You definitely want to check those out. So that's it for this episode of the Test Guild Performance and Site Reliability Podcast. I'm Joe and my mission is to help you succeed with creating end to end full stack testing awesomeness. As always, test everything and keep the good Cheers.