Develop JMeter Scripts Using Java with Roger Abelenda

Welcome to the Test Guild Performance and Site Reliability Podcast where we all get together to learn more about performance testing with your host, Joe Calantonio.

Hey, it's Joe. And welcome to another episode of the Test Guild Performance and Site Reliability Podcast. Today we'll be talking with Roger all about JMeter JMeter, Java DSL, a really cool development that I just saw recently on a perfbytes twitch, so I thought you all would want to know more about it. If you don't know Roger is a software engineer with more than 15 years of experience in development with a special focus on developing high traffic and resilient applications, which is really critical nowadays. And in his career he's played different roles such as a developer, architect and technical director. Currently he's the CTO at Abstracta, a software quality focused company. He uses his experience and technical background to help different areas of business, fostering innovation, and always willing to help, which he's joining right now to help us learn more about java, more about JMeter and how we can programmatically create performance scripts. So really excited to have him on the show. You don't want to miss this episode. Check it out.

This episode is brought to you by the awesome folks at SmartBear. Listen, we know load testing is tough but necessary. So investing in the right tools to automate tests, identify bottlenecks and resolve issues fast saves your organization, volt time and money. And that's why SmartBear created Load Ninja, a SaaS load testing tool to help teams get full visibility into performance so you can release quality software faster than ever. Make testing effortless and give it a shot. It's free and easy to try. Head on over to loadninja.com and learn more. Hey Roger. Welcome to the Guild.

Hey, thank you.

Awesome to have you. So, before we get into it, is there anything I missed in your bio that you want the Guild to know more about?

No, I think that's fine. I mean, it's more or less what I'm. It describes me good enough.

Awesome. So I guess before we dive really into more detail is what is this DSL, this JMeter DSL?

Well, the shade meter DSL is just thin layer, like it's an API, a library that you can use in Java and you can use for designing and running Shade Meter. Embedded Shade Meter. You don't have to install Shameter or anything like that. You just use the DSL and it allows you to easily create test plans and run them. You don't have to actually have deep skills on programming or Shameter. Itself you have a user guide. That's basically what we are trying to do is shift left in performance testing. So making performance testing easier and sooner in the development cycle of the development itself.

Nice. So I guess why though is it because JMeter for a newcomer, sometimes when you look at it seems overwhelming. So does this, like you said, was created to make this process easier. So like you said, you could actually get your developers more involved earlier?

Yeah, that's basically it. I myself, as you described, I was a developer working a lot with high performance applications. Early on I started using JMeter which was the most popular tool. So when I start using it identified, I was kind of not comfortable enough doing with the current UI that it has. That is awesome. I mean it's a way to start doing performance testing. But when you are starting to do when you want to program your test or you want to share your test and modularize it better, it start to falling kind of short or it's kind of get complex. When doing it in a parametric way, it will be easier to manage at high scale. So that's why one of the main reasons that we decided to do this and, and there was no like Java native solution for doing this. So we started doing this and after that some other solutions came up. But yeah, I mean that's the main focus on why we decided to do it.

Nice. So I'm very familiar with K6 and familiar with Gatling. Gatling, they all use this programmatic type of approach. Is this the same? Are you developing a Java JMeter Scripts within Java just typing up the commands?

Yeah, it's kind of the same. I see K6 as more JavaScript oriented or more like maybe front end if you like. So if you already work with JavaScript or maybe use Node JS for doing your backend servers, probably you will be a lot more comfortable with K6. Gatling requires a Scala environment, but they later on, after we released the first initial version of the Shameter dsl, they also released a Java DSL which is more comfortable to use for Java developers. But still it is Scala underneath. So you still, if you start debugging or something like that, you will get that. But on top of all that, when you use Gatling on K6 you lose all the ecosystem popularity, your knowledge about Shameter and the tools around it. So it's kind of a bummer to have to stop using JMETER to start using Gatling just for the programming sake. I mean JMeter has a lot of flexibility, has a lot of support for protocols and Again, an ecosystem that you will lose if you move on to Gatling or something similar. So that's like comparison between using Jmetershava DCL and using Gatling or K6, right?

Yeah, and a lot of people already know JMeter but like you said, it's like almost. There's a high overhead to get up to speed with it. And also it seems like this would make it easier almost to fit into pipeline CICD subsystems. So is this another advantage of this approach?

Yeah, sure. I mean totally. We the way we Encourage using the JMeter DSL is using Junit or testng or any other testing framework that you use. Just there you put your performance testing using JMeter DSL, it will just run with Junit or Testnshi so it will automatically be supported by any CI CD tool that you use and you will got all the reports that you usually get with chainit or Testng and you don't have to actually install anything. You don't have to install Shameter on the machine or nor you have to install some or know how to invoke some command line tool. It just is automatically integrated with any of the framework that you usually use for testing.

So what do people need to get started though? They just import the library and then do they point to a machine that has JMeter on it? How does it work behind the scenes?

Yeah, in fact you don't have to have a JMETER machine to run the test because how it works, you import the library, the library will automatically also import JMETER libraries underneath. So, so the JMETER core and it will use JMETER to run the test and to execute the text itself. If you want to run the test at scale. For example, if you want to run in a cluster of machines that you already have with JMeter, you can do it with the library. There is an embedded, sorry, remote engine, stuff like that. You can go there to the user guide and check how you can do it. But you can also use for example one of the service on the cloud JMETER execution engines like blazemeter Octopurf, we have adapters for those two already built in. But if you want to, I don't know, use another cloud service, let us know and we can implement some engine for that. Currently the DSL we are open to all suggestions. So currently we are asking people to request things and to come up with ideas for improvement because we are trying to create the best tool that we can out of this solution.

Yeah, absolutely. So we'll have a link to maybe drop you a note or something for any suggestions directly for. Because I think it's a great development. How long has this been out? Like I said, I just saw it with Mark Tomlinson and you did Twitch with them, but I don't know how long it's been around. Is it fairly new?

Yeah, the first release that it was kind of a proof of concept was in August 2020. So that was the first release and we have come up with releases from then on. Like one release a week kind of. Because some weeks we do several releases, some others we don't do any. In fact, the approach we are using for making releases and including features on the DSL are by people asking us things. We try not to implement things. We have a lot of things or ideas that we'd like to implement, but we wait for someone to ask the feature because we don't want the tool to be kind of over complex and don't have features that you might not use that we think that Shimmeter has come up to such a point that sometimes you don't know when to use one feature or when to use the other and sometimes you might use whatever of the features. We try to come up with the reason for each of the features and we document such research in the user guide. So the idea is in the user guide we say, okay, if you want to do this, you can use this. This is the best practice. We embed in the DSL all the good practices that we have identified on the usage of JMETER in Abstracta. We have a performance expert area that we have worked with Shameter for several years. So we have accumulated a lot of knowledge. Every design or every decision that we make to implement a feature will based on those best practices and coordash users to use such good practices.

So I guess that was kind of my next question. How did Abstracta get involved in this then? Is it built from a need from your company? And you're like, wait a minute, we're getting a lot of customers that can benefit from them. Let's open source it.

Yeah, I mean I had this idea when I was not working at Abstracta, I was in another company and I contacted the CEO of Abstracta because I knew him and said, hey guys, do you know of any tool that can do this with JMeter and say, no, I don't know any. So I was working with JMETER for some years. Then after several years I moved into Abstracta and when I move into Abstracta I kind of saw the same or kind of suffer the same pains and I saw Also the same pains on other teams in the in Abstracta. So I decided to come up with okay, this is enough, I mean, we have to do something about it. And I did a proof of concept over a weekend. I show it to the CEO and the CEO said, yeah, this looks great, let's put it like open source software and see if the community gets any attention, if they like it. And that's how we did it. And we keep investing on it as we see that the community is interested in the solution and is collaborating and asking things. So that's the way we are working on it. We don't want it to be kind of an abstract a thing, we want it to be a community thing. So we want to involve other companies helping us developing this, the entire community, the open source community. So yeah, that's kind of why we came up or why we invest on it. Why is open source by itself? We think that is, I mean open source is the best way for getting the best solution. We want to not only if you know Abstracta or if you kind of research about Abstracta, we collaborate with a lot of papers, we exchange a lot of knowledge that we have accumulated during the process of. I mean through our expertise or our experience with clients, but we also implement software, I mean solutions that is our processes. Shimmeter DSL is one of these solutions. We have open source, some other solutions. For example, there is a virtual service for TCP that is named wireham and from the very beginning of Abstracta, the Abstracta was born out of a product abstracta implemented that was a testing solution for genexus that is a particular software or framework for creating software in a codeless solution. But yeah, local solution. Sorry.

Nice. So as we said Jamie, it's been around for a while and it seems like they've added onto all these different plugins. You could do like the UI is. You could do all kinds of things. So there's anything missing from the Java DSL that if people need to be aware of their hardcore JMETER users already?

Yeah, sure. I mean as I said, we haven't implement everything that is already built in JMeter and we wait for people asking things before we implement them on the dsl. The main or the core logic or the things that are most used by people are already built in. For example, we have IF controllers while for each regex extractors, CSV dataset, HTTP requests, headers, cookies, etc. Like the core thing is already in the DSL. But there are a lot of things Doshimito Already provides for example, it has support for RT protocol Citrix WebSockets HTTP 2. I don't know a lot but we haven't implemented them yet if you ask for them or we can easily implement them. In fact, the process of implementing new features in the DSL is quite easy because we just kind of come up with some way to create these elements or this JMeter elements and provide a builder or a method that makes them easy to use. The part that takes more time is like the analysis of what things we should include or not in the elements but the implementation itself, the code is quite, quite simple. So yeah, usually we try to. If someone asks something, we try to come up with a solution as soon as possible. If we think that it's something that entire community will benefit from it or it's a common use case. If it is something too much specific, we might tell the people, okay, maybe you can create something or we might provide some solution for that that is not maybe incorporated in dsl but in general we try to cover all the needs of the users. But again, there are a lot of things in JMeter we don't want to cover everything at the beginning and we want to build as people need them.

I guess the opposite question then there's so much in JMeter but is there anything missing in JMETER that you think would be added to this JMeter DSL?

Ah, sorry. Yeah, we added some things for example execution on the cloud environment, for example Octoparf and Blazemeter are things that are not like built in on JMeter or you have to come up with ways to do it. So we have built some things built in on the dsl. Another thing, for example we come up with a kind of a live dashboard that mixes some of the charts. Maybe you already saw in the Perfbyte demo that mixes some of the charts in JMeter and show them just in one view and you don't have to kind of go through all the charts. I don't know. There is for example an integration that is there in the user guide and it's in the repository about user influxdb and Grafana with the Shamemeter dsl. So you don't have to, if you want to set up that you already have them set up there about best practices. For example in Shameter it's quite easy for example to do a post without putting a content type, which is usually, I mean if you send a post to a server, usually you have to send a content type. So we have already in the dsl whenever you use a post you kind of not force but in the easiest way you will need to put a content type. You can skip over that if you don't want to put a content type. But in general these are kind of the best practices and things that we have built in on the dsl we also have implemented an RPS throughput controller that is just a mix of different components. For example, we mix the throughput shaping timer with a concurrency thread group and we put that just in one thing so it's simpler for users to use. The same goes for example with random CSV data set that you don't have to say okay, I want to use this plugin just for the random order. You just use the CSV data set and say random order. And underneath the DSL we'll use the proper plugin or the proper extension for that Particular features, those are kind of the things that we simplify and are easier and built in on DSL. While if you want to use JMeter you will have to do like some extra effort or some extra thing and those things are also things that we are looking for. I mean if users see that something is a common pattern that which I meter might be difficult or requires several steps, please come up with ideas and we will implement them. Some of the ideas that we have, we have plans later on to contribute them to Shameter but some others don't make much sense on Shameter themselves because are kind of things that are inherent on the usage of the of JMETER inside code. Like it makes them make it easier to use Shameter and you will need it in code but in the UI they don't make much sense. So there are things that we might contribute to Shameter, some others that make not much sense.

So it sounds like you've been. It's been available for almost a year now. Next, next month we'll make it a full year. So have you seen any weird use cases?

Two years.

Oh, two years. Oh, 2020. I was thinking 2021. So two years. So have you seen any weird use cases of people using it in a way that you. Wow, I didn't even think you could use it in that way. Like I know people because it's just a library within Java can is anyone using it with Selenium, doing a call with JMeter and then using the data and with their automation frameworks or anything like that?

Yeah, I don't know if weird it's the gory world but we have seen like very interesting use cases. For example, we have seen, we have been talking with Some teams on Russia that they use, they have created their own DSL on top of the dsl. They have created kind of simpler way of creating performance tests without having to specify thread groups and the like. They haven't shown me the solution, but it seemed really, really nice. It was like some kind of codeless solution for performance testing, something like that. That was very interesting, but I haven't seen the final product or any demo yet. Some other use cases that I have talked with users, there have been two users, two separate users that have integrated kind of a functional way, like functional test with Rest Assured. They implement the test with Rest Assured or Cucumber or something like that. And then they use the DSL to convert those tests to performance tests. And they have kind of an architecture of test. I mean it's really interesting and I don't know if they will be doing that with chameter. I mean I guess that the DSL as it's easier to program or easier to integrate with programming features, make it easier to build things on top of it. It's kind of okay. I have this base block. I can implement things and integration more easily than doing it with the JMX of JMETER or actually the code of JMETER itself.

Yeah, because you don't actually need JMETER on the machine, you could just run it in containers easily. So it seems ideal for that kind of a user case for sure. Are there any gutches though when people do this? I mean do you use this for an enterprise wide type of performance test? Is this more just for like a smoke performance test or like an initial performance test before you do an official large scale traditional stress load test?

Yeah, I mean by default when you put on the test plan run it will run an embedded engine and you will be limited by the engine itself, like the load generator itself capabilities. Usually when you want to to run a enterprise test, you would like to do it on a cluster of machines which you are not tied to the limitations of the hardware or the server where the JMeter DSL runs itself. So that is kind of the gotcha or care that you have to do. I mean you can run it with the Shastron with the embedded engine. But usually for running really interesting performance tests or low test, we'll probably want to use a cloud, sorry, a cloud service. Aside from that there are not any gadgets. I mean it's just running performance JMETER test. All the gadgets that you might have with shame meter might be gadgets. On shade meter DSL you need a shade VM which Is pretty standard and very, very common on any CI CD or any environment to run any tool. But I don't know any other gotchas. I mean about what you meant like long term maintenance or maybe enterprise performance testing. It is easier to modularize or to reuse part of the DSL than using jmx like in shameter. Shameter. Usually to modernize a test plan you have to use include controllers with fragments or create different JMX files which they by themselves don't make much sense. Or variables. You have to come up with variables. But with programming it's usually easier to do that. You can create abstractions with classes or functions that you reuse for creating different parts of the test plan. You can put parameters to parameterize this test. I don't know. It's in the long term, I mean for creating very complex tests, I think it is easier or simpler to use a programmatic way than using the JMeter GUI, which might be kind of difficult. Also visibility of the test plan is easier on the dsl, on the code, because you see everything that you have configured, everything you have set. And on JMeter, to get a full view of the test plan, you have to go over each of the elements and each of the tabs to see any advanced features. And you have to already know what are the default ones and what are the things that the user might have changed on that test plan. So it's more difficult to understand what is the full extent of a test plan in JMeter, I think than in the DSL. You can even in the DSL, if you modularize the code, use Codenavigator on the IDES. All the IDES features like auto completion, built in documentation are already there. Like all the features that usually programmers have and you kind of lose while using Shameter because Shameter is a UI and it's not an IDE. I mean, the IDEs have evolved a lot and they are pretty smart about that. So we kind of reuse all the smartness or all the evolution of IDEs in the DSL and benefit from using it.

We keep going back to it fits better into the ecosystem because it fits into your CI CD systems. You could do code reviews, I guess you could check it in just like all your other code, all the other assets, they all live in the same location. It just seems like a better, more natural flow if your developers are going to be contributing along the way.

So yeah, yeah, sure.

Cool. I know with JMeter a lot of times they have there's a JMeter recorder, is there Something similar with this or someone just programmatically straight typing out what they need to do.

Yeah, right now if you want to use the recorder or recording approach, what you can do is is use the JMETER recorder or the Chrome extension that Blazmeter or any of the solutions that you have to record. There is a blazemeter extension that you install in the Chrome browser and then you can record and you download the jmx so you can use that as well. And while after you have the JMX either recording with jmeter or this Chrome extension or any other solution, maybe you have a har file that you created on your browser and you converted on loading or some other solution that has this kind of conversion software. Once you have the jmx, the DSL provides a tool that is named jmx to DCL that automatically converts the JMX to DSL call and then you can start working with the DSL call from then on. You can also go back to the JMX if you want. I mean there is a save to JMX method that you can use if you want to, I don't know, continue editing the JMX on Shame meter. Or maybe there is something DSL that is still not supported by JMeter and you don't want to get stuck until we do something about that. You can save to JMX and continue working with JMETER if you like. You have all the full cycle like go back and forth from shamex to dsl.

Okay Roger, before we go, is there one piece of actionable advice you can give to someone to help them with their performance testing efforts and what's the best way to find contact you or learn more about Java dsl?

Yeah, I think that part of it is what I already mentioned, but I think that the best thing that like the best action is go to the user guide, check it and invest in it. Try the dsl. It's just downloading, it's just putting a dependency on your maven project or gradle project and use it if you already have Java. If not, it's pretty easy to install Java and start working. In fact I have tried it with tool that is shebang. I don't know if you know it's kind of a new tool, but it's a nice tool that I have in Discover that allows you to run Java code just from the command line as if it were like a groovy script or kotlin script so you don't have to actually compile or use an IDE to run it. So it's pretty awesome. Yeah, to contact me you can follow me on river on Twitter and you can direct message me. My account is Rabelenda. Also you can contact me via LinkedIn if you want. Roger Abelenda and yeah, we can talk about that. We can also you can contact me also in the Discord Channel, labor, she meter, Shabbat dsl all with hyphens in the middle and I think that's basically it.

Thanks again for your performance Testing Awesomeness. Links to everything of value we covered in this episode. Head on over to testguild.com P93 and while you're there, make sure to click on the Try them both Today link under the Exclusive Sponsor section to learn all about SmartBear's two awesome performance test tool solutions, Load Ninja and Load UI Pro. And if the show has helped you in any way, why not rate and review it in itunes? Reviews really do matter in the rankings of the show and I read each and every one of them. So that's it for this episode of the Test Guild Performance and Site Reliability Podcast. I'm Joe. My mission is to help you succeed with creating end to end full stack performance Testing awesomeness. As always, test everything and keep the good. Cheers.

Thanks for listening to the Test Guild Performance and Site Reliability Podcast. Head on over to test guild.com for full show notes, amazing blog articles, and online testing conferences. Don't forget to subscribe to the Guild to continue your testing journey.