JMeter and How to Debug Test Scripts with Naeem Akram Malik

Foreign.

Get ready to discover some of the most actionable performance engineering, testing and site reliability device with some of the world's smartest engineers. Hey, I'm Joe Calantonio, host of the Test Field and SRE podcast and my goal is to help you succeed in creating application performance awesomeness.

Hey, it's Joe and welcome to another episode of the Test Guild Performance and Site Reliability Podcast. And today we'll be talking with NAIM all about JMETER and how to debug test scripts. If you don't know, Naeem is a experienced test engineer, a very prolific Udemy instructor and technical writer. I think he has over five courses on Udemy like TCP, IP socket programming with C sharp, net, Appium, WinApp, driver UI automation testing, Windows UDP Socket Programming, I think for distributed computers, Windows Service programming, debugging, Azure, DevOps, building pipeline. So he has a ton of experience, a ton of students, and I think he has almost five star ratings of all these courses. I've been wanting to get him on the show for a while and so I recently saw a blog post he wrote for blazemeter and I thought it'd be a great opportunity to get him on. He also has a lot of hands on experience in automation, web testing with Selenium, API testing with C Sharp and Java, as well as performance testing with JMeter. And also make sure to check out the show notes at the end of the episode to learn how you can actually work with NAIM directly. If your company is looking for a remote association or remote assistance with qa, automation or performance. So you don't want to miss this episode, check it out.

This episode is brought to you by the awesome folks at SmartBear. Listen, we know load testing is tough but necessary. So investing in the right tools to automate tests, identify bottlenec, resolve issues fast saves your organization both time and money. And that's why SmartBear created Load Ninja, a SaaS load testing tool to help teams get full visibility into performance so you can release quality software faster than ever. Make testing effortless and give it a shot. It's free and easy to try. Head on over to loadninja.com and learn more. Hey Naeem, welcome to the Guild.

Hi Joe, I'm very happy to be here today. Thank you for inviting me.

Awesome to have you. You have a lot of experience, but is anything I missed in a bio that you want the Guild to know more about?

Well, no, you actually did a great job. I mean I got a lot of stuff on my hands and it's like all over the place. But yeah, a lot of stuff is going on. The technical writing, the test automation, the udemy courses, the YouTube channel, whole bunch of stuff and that was a nice intro. Thank you very much.

Awesome. So I guess how come there's no course on JMETER yet or performance testing?

Well, honestly speaking like just a few months ago, until a few months ago I was, I never used JMET for professional purposes. I was mainly focused on UI test automation with Selenium and before that it was coded UI a long time ago. I know you remember the name but I'm quite a dinosaur, you know. So I got, so I got a chance to work on JMeter just a few months ago and that's when I actually fell in love with the technology. It is like really wonderful. So on my day job I got the chance to like explore JMeter because I had to do some like work related stuff. So I got introduced to JMeter and then I started learning about it and ever since I have been working on it and I have been thinking about how to prepare a course about JMeter. I have drafted an outline and I'm working on it. I would like to publish one course before the Black Friday. So I'm working on it and I hope I will be able to like do it alongside the blog writing because I have been writing a bunch of blogs lately so that's an actually a new thing. I have been blogging since 2009 but the paid blogging gigs or the pro like blogging gigs have came a little lately and I'm happy about that. But yeah, the course is also in the making and you could expect it in a couple of months.

How exciting. I will promote it once it goes live, so just let me know. That is awesome news.

Oh my God, that would be great.

What I love is, what's even better is you said you're kind of new to JMeter. So how did you learn JMETER then? How did you get up to speed? Because it wasn't an awesome class. Like you're about to create. Like how did you struggle and how did you learn on your own?

Yeah, basically there are a whole bunch of classes. So first of all, before this in February last year, I watched a class. I had a pluralsight subscription. Although I'm teaching courses myself but I learn a lot of courses. I don't teach as many courses. I learn a lot of courses. So I watched a bunch of courses from pluralsight and when I had to like, I got a training from a colleague at our company who was also like experienced in JMeter. And after that I didn't get a chance to use the stuff which I had learned and like had to put it on ice for a few months. So first of all it was the pluralsight course, then an in house training and then I came to this new place where I had to use JMeter for all practical purposes and I thought, oh my God, what should I do now? So I ran around and I found another course online. I think it was on LinkedIn Learning. Yes, I found a course on LinkedIn Learning. I've watched it, but still there were a lot of unanswered questions. And then I went to JMeter's very own documentation for like finding answers related to the things which I needed to do. And then there was this Blaze Meter blog which is also very nice resource. You can check it out. There are a lot of topics, they are covering them very nicely. So these were like. It was not one source, it was like plural site. LinkedIn Learning, YouTube, Blaze Meter blog and I don't know what else. Yeah, a lot of things. So it took me a while to like understand it and luckily I had the time, my employer gave me the time to explore and to come up with a solution to learn. So I was like learning on one side and then implementing it practically on the other hand. And that's how I like got into the groove, you know.

All right, so you know you mentioned how you had to learn JMeter for your job. Did you also have to learn performance testing at the same time or did you already know about performance testing and it was just a matter of learning JMeter?

I knew a thing or two about performance testing and I had to learn about it as well alongside JMeter. And in fact I think that I'm not like an expert in performance testing. I know a thing or two, but I'm still learning.

Cool. So it seems like based on your Udemy courses and I could be wrong is you're more of a C programmer. Don't know if that's correct or not, but if it is, why would you choose JMeter? Are there anything that works with C Sharp that would have been a better choice for performance testing?

Yeah, I have a lot of experience in JMeter, but last year I changed my technology stack to Java. So I have been working in Java since February of this year, February 2022, I have working with Java completely. I had past experiences with Java as well. So it was not a problem. Switching the tech stack was not a problem. So I switched my tech stack like earlier this Year. So that is one thing about my experience. I got a lot of C Sharp experience and I also got a lot of Java experience now. And plus, you know, the both languages are similar. There are a lot of similarities. So it was an easy transition. That's one thing. And about picking the JMeter, basically, yeah, there is nothing else like JMeter out there when it comes to the dotnet platform. There is Postman we now like for API testing, but it cannot be used for performance testing. So there is no tool available which works only based on C. So that is why I crossed over to Java and then I picked up JMeter because I believe it's the best free and open source tool out there for performance testing.

Perfect. So I thought we'd start walking through like talk through basically your latest blog post that you actually did write for Blazemeter, I believe on how to debug JMeter test scripts. And so I guess the first part was you have a section on how can teams debug JMETER test scripts. Why did you focus on debugging JMETER test scripts? Is that something you found as something that was difficult part of JMeter?

Yeah, that is a very important thing. Well, the debugging is a very important part of, you know, programming and any programming exercise in software development. Debugging is very important because we always make mistakes, people make mistakes, intentional, unintentional, for whatever reason. So debugging the JMETER scripts can be tricky because it is a little different from your common scripts that we write. So it is a tricky thing and I believe a lot of people need help in this regard because I myself faced some problems related to debugging where I was scratching my head while executing my performance scripts and I didn't know what was going on. So I had to learn about debugging. And then I figured out that okay, this is how it should be done. And then came up the opportunity to write about this. Blazemeter asked me if I could write about this and I jumped on the opportunity. So that's why I wrote it. I really believe that it's an important topic.

Nice. So you know, I'm used to performance testing using Loader and I actually never really used JMeter. So, you know, there's some things in here I'm learning myself. And the first one was samplers. So I guess what's a sampler? Because it always seems like everything has a sampler and you have a section on JMeter debugging sampler. So debugging is really important. So what did you find challenging about debugging in JMeter that you thought I need to write about it.

Yeah. So first of all, if you are writing code using C Sharp in VL Studio or in like intellij idea, we can simply put a breakpoint anywhere in our script and as soon as the breakpoint gets hit, we can see the values of various variable like other important things by like hovering the mouse and like, you know, so in case of JMeter that does not happen when we are creating JMETER scripts and we make a mistake. So we don't get that luxury of the breakpoints. So the breakpoints are not an option for JMeter. So I had to like condition myself and I had to learn that how I have to live without the breakpoints. So the breakpoints, you know, come in very handy when I was working on Selenium, like in C Sharp and Java. So that was the main difference for which I thought that, okay, this is an important topic and I should write about it.

Absolutely. So is that where the JMeter debugger sampler came in into play? Like, what is the JMeter debug sampler?

Yeah, so the JMeter debugger sampler is like the core of the debugging functionality which is provided by JMeter. So that is where the JMeter debugging sampler comes into play. It allows us to see the values of different variables which are used inside of performance testing scripts on the runtime. Like we can run our script and see what different values at different times. So that is how the debug sampler is helpful. And it can be compared to like, it can't be compared to the like breakpoint, but it is somehow related, relevant to the breakpoint when we are like using Real Studio or Java. So we have like breakpoints in IntelliJ and we have Studio, but we don't have breakpoints in JMeter. So the debug sampler is kind of an equivalent. It allows us to see the different variable values in our performance testing scripts. So that is basically where the debug sampler comes into play. Yeah.

How does that work? Is it like a print statement or does it like at the end you have a log file and it just prints out like I did this, this. It was the value I sent. Here's what I got back.

Yeah, it works like a print screen essentially. So even have, I believe a script on my screen, I can share my screen to show it to you as well if need be. So yeah, it works like an all inclusive print string which is going to print all the variables which are being used inside of performance testing scripts at any given time, like wherever we place it.

Nice. So is that where you use the JMETER debug post processor? Does that help you read that after the fact then?

Yeah. So the JMETER debug post processor has got a different scope of operation. A post processor is essentially placed inside another like samplers. So in JMeter we have different samplers. A lot of things are organized into samplers. So the debug sampler and the debug post processor have a main difference that a sampler is like a standalone thing and the post processor is like it comes underneath any other sampler. Like it comes underneath an HTTP sampler. For example, if we have an HTTP sampler or a TCP sampler. So we, and we want to see the variable information which is specific to that sampler. In that case we should use the debug post processor. So we can say that the debug sampler has got like a very wide scope and the debug post processor has got a narrower scope.

So I guess then the next question that you cover is what's the scope of the JR223 sampler?

Yeah, so that is like an additional thing. So it is. Again, the scope of the JSR 2 to 3 sampler is same like the debug post processor, but it is not created for the debugging purposes. When I wrote about it in my blog, I was essentially reusing it for my own purposes. So the JSR223 can be used to write script in multiple languages, like in JavaScript. In Java there is groovy support. So the simplest use for debugging could be that if you want to print only one variable in the log, if you want to print only one variable in the JMETER log, then we can use the JSR223. I hope I'm not botching the number. It is like tricky. 223 or 2, 3 2. But there is like this code execution controller code execution sampler in JMeter which allows us to run a lot of programming scripts. So I repurposed it for like debugging. Oh cool.

So how did you come up with that idea? You were just messing around with samplers. It's like, oh, I can reuse this to help me with debugging. Or did you find that tip somewhere else?

Yeah, you know, old habits don't over easy. So I have, I have been like doing like coding. I have been writing code for such a long time. I have been like using print screen and like test or debug all those statements to print stuff freely. Like whenever I would want I would just like throw, throw a bunch of like your print print lines out there and, and the text would show up in the log in test automation logs in my selenium logs. But now here I was in a situation where I couldn't just like freely put anything out there. So I thought what could I do? And this was the idea which came to my mind. Okay, let's do it my way.

Cool. Awesome. So how does JMeter work? I know like I said, I have experience with Load runner and it's like an enterprise solution that has analysis reports, you can monitor things. GME meter to me sounds like just something that does the load. But like how do you like, does it have a reporting feature and how do you analyze different meters like CPU and memory usage using it?

Yeah. So like JMeter is basically an open source tool so it has got a lot of plugins around it. LoadRunner, UFT LoadRunner. It is a great tool definitely and it is a premium tool. So JMeter is basically trying to bring all of this together using the plugins. So we have like a lot of plugins and we have a lot of like built in support as well. For reports. For example, first of all for generating a report, if you want to see the test and report, the JMETER is going to generate a performance test report which can show us the APDEX index of appdex is basically a metrics which is used to track the performance of various APIs and web pages. So the JMETER by default has got the support to generate the AppDext index in an HTML file so we can configure it to generator an HTML report which shows us the latency of different requests, which shows us the average time of different operations like HTTP get, HTTP post and all those. It also shows us if there are any errors in there and then we can configure it. We can configure JMeter to work with other tools like Perfmon for example on remote servers. And there are other like SQL monitoring tools as well which can be configured to work with JMeter and using these we can easily create a report which suits our purpose. So basically JMeter uses the plugins and other things around it like Lego pieces. Piece together everything you you need. Yeah.

How do you find them though? Is it like a library or somewhere you go that list out everything? I guess like I said, JMeter does have good documentation. Were able to find all these. Is it just A matter of doing Google searches to find all the different ways you can do that.

Yeah, JMeter has got a plugin search functionality built in. So there is a JMeter plugin manager. So we can use the JMeter plugin manager. It does not come installed with JMeter out of the box, but the installation is very simple. We just need to copy, download it and copy it over. So once we have the plugin manager we can use it for searching any plugins. And there is a plugin directory online as well where we can find the plugins which are available and we can download the plugins as well. Although if we are like setting up new machines then it would be like our own responsibility to take care of the plugins. Otherwise the scripts could break. They don't always break, but they could break. So that's how plugins work in JMeter. Cool.

So this was posted on Blazemeter. So I guess what is Blazemeter? Did you use Blazemeter? Does Blazemeter make using JMeter easier? Like maybe a little bit more info about what is blazemeter, if you even use it at all. If you just blog in there.

Yeah, I have done a little bit research about blazemeter. I have not used it on in my personal projects until so far. So I just stored them a blog post about JMeter. But as far as I know Blazemeter at first used to be all about creating JMETER scripts easily and running them. You know blazemeter provides a recorder which can be used to create JMETER scripts very easily through your browser. This functionality does not come built in with JMeter. So blaze meter, they innovated in this way that they provided a recorder which you could use through your browser to create jmx. JMX are the like JMETER files. So that is one innovation by Blaze Meter. But they have been like out there for more than a decade I believe now. And they now have rebranded themselves as continuous integration platform, a full blown CICD platform. So they started from JMeter but now blaze Meter has like grown into a lot of things. But I haven't used all of them like so I can't really give you like a marketing brief about it. Yeah.

So what were you testing? Were you testing APIs? Is that what you originally said what this project was about?

Yeah, well that's the interesting part about JMeter. The general concept about JMETER is that it is used for API testing, but it can be used for website testing as well. So when I got to this new job, it's a small software house, a small like software shop. It's called graymath. So when I got to gray math, what happened was that they were testing the performance website which they are creating for a client, actually a very big application in dot net. So they were opening the different pages of this website multiple times, four, five or ten times. People would sit, open the pages and then note down the time from the network monitoring tab in the Chrome browser. And then they would put it in an Excel sheet and then based on that, they would calculate the averages and other things. So when I got there, I saw an opportunity. I did this activity a couple of times, but then I saw an opportunity that we could do this with JMeter. So I like started working on that. I told them that I believe we can do this. And they told me, yeah, you think we can do this? Go ahead, let's do it. So they're like good people, innovative people. So they gave me the space to innovate. So I went ahead and I like started like automating all of this process using JMeter. And along the way came many challenges which I was luckily successful to resolve. So that is how it started. I am testing a website using JMeter. Yeah. The performance of our website. Yeah.

What were some of the challenges then that you ran into?

Yeah, yeah. First of all, the challenge was that how could I, like extract the cookies? Like, so first of all, like, what to do about the cookies? All right. And then I came to know that, okay, it is easy to extract the cookies. So I found an article about it on, I think it was a stack overflow post. So I found about it and I used whatever I found in there. I put together, repurposed it for like my own thing. So I got to use it. And then there was another challenge that like when I was creating a new item using this website, a unique ID was generated and I didn't know how to extract it. I had read about the regex extractors, the regex post processors in JMeter, but I had never used them in practice. So I like, first of all, I figured out like, I should use the regex post processor. And then again I got stuck with the regexes and then I watched a course about regular expressions and then I like took a day or so and then. But I got very good at it eventually because I have some background with the regexis. So these were a few challenges which came my way. And then generating the report was also a challenge because I wanted to have something which I could send to the stakeholders. I couldn't send them the screenshots of the viewers of the JMeter every day that, okay, this much time was taken by this call. Etc. Etc. So what I did was that I next thing I figured out how to generate reports using JMETER through the command line. And JMeter allows us to generate reports through the command line and they are in HTML format. So it's an HTML directory which we can use, which we can easily send to our stakeholders, which you can put on like a CI CD server anywhere so that everybody can see it. So these were a few challenges. Yeah. Great. And there are many more. I would be. I'm here to talk all night. No problem.

Cool. Okay, before we go, is there one piece of actual advice you can give to someone to help them with their GMETER performance testing efforts? And what's the best way to find or contact you?

Yeah, well the advice I would like to give anybody is that first of all so the advice I would give someone who's starting like freshly with Jim, it is that take it easy. Don't worry, there are a lot of resources out there. It is only a matter of time when you will be able to find them and when you'll be able to put together your own solution. So you can start with good courses. There are many good courses out there already like rural site on LinkedIn Learning. There are very good courses already available. If you can like buy some then I highly recommend like investing in yourself. So that's one thing. And then we can like easily, slowly and gradually. Yeah, the more important thing is that if you are starting on like a greenfield project, what they call it, if you're on a green field project, then you are going to piece together things slowly and gradually. So I highly recommend that you start learning from like an online course and then you start consulting the documentation provided by JMeter. And then there is this Blaze Meter blog. They also have a lot of useful things. So we are going, you should go step by step and build your performance testing scripts slowly and gradually. And like once you have something which is useful then you can refine it further. So take your time, stay patient and build it piece by piece and ultimately you'll get there.

Awesome. And the best way to find or contact your name?

Yeah, so best way to find or contact me is first of all my LinkedIn page. Please share the link. I can be reached out to my LinkedIn profile. You can drop me a message over there. You can also contact me through my email or I have this website, test automationtv.com it is going to take you to my test automation course for now, but you can find all of my social links from there too. So there's my LinkedIn profile, there is my email, and then there is also my blog. My blog. A programmersday.blogspot.com A programmersday.blogspot.com if you go there you will find a Contact Me form and you can send me an email through there as well. I've been writing there since 2009 so and that's how I discovered that I have a knack for teaching. A knack for like showing off my work. Yeah. So these are a few ways that you can contact me.

Thanks again for your Performance Testing Awesomeness Links to everything of value we covered in this episode. Head on over to testguild.com P97 and while you're there, make sure to click on the Try them both Today link under the Exclusive Sponsor section to learn all about SmartBear's two awesome performance test tool solutions, Load Ninja and Load UiPro. And if the show has helped you in any way, why not rate and review it in itunes? Reviews really do matter in the rankings of the show and I read each and every one of them. So that's it for this episode of the Test Guild Performance and Site Reliability Podcast. I'm Joe. My mission is to help you succeed with creating end to end full stack performance Testing awesomeness. As always, test everything and keep the good. Cheers. Hey, thanks again for listening. If you're not already part of our awesome community of 27,000 of the smartest testers, DevOps and automation professionals in the world, we'd love to have you join the fam@testguild.com and if you're in the DevOps automation software testing space, or you're a test tool provider and want to offer real world value that can improve the skills or solve a problem for the Guild community, I'd love to hear from you. Head on over to test guild.info and let's make it happen.