PAC Hero Festival with Henrik Rexed

Welcome to the Test Guild Performance and Site Reliability Podcast where we all get together to learn more about performance testing with your host, Joe Calantonio.

Hey, it's Joe. And welcome to another episode of the Test Guild Performance and Site Reliability Podcast. Today we're going to be talking with Henrik Rex all about the Pack Hero Festival, which is a free trial, 24 hour performance engineering and testing online conference that has taken place March 23rd to the 24th. I really believe there's a need for more education for performance testers and this is definitely an event that's going to get you up to speed on some of the latest and greatest trends in performance testings. And for me, I think this is a must attend event and we're going to talk a little bit more about in this podcast to let you know why. All the reasons why you should join the this year's pack if you don't know. Henrik Rex is a performance engineer at neotis, a leading provider of load testing software. He has been orchestrating and conducting performance tests for well over 15 years, delivering projects in all contexts including extremely large cloud testing on the most demanding business areas such as trading applications, video on demand, sports websites and more. He's a thought leader in this space as well. You don't want to miss this episode. Check it out.

This episode is brought to you by SmartBear. Listen, load testing is tough. Investing in the right tools to automate tests, identify bottlenecks and resolve issues quickly could save your organization time and money. SmartBear offers a suite of performance tools like Load Ninja, which is a SAS UI load testing tool, and LoadUI Pro, an API load testing tool to help teams get full visibility into UI and API performance so you can release and recover faster than ever. Give it a shot. It's free and easy to try. Head on over to smartbear.com solutions performancetesting to learn more.

Hey Henrik, welcome back to the Guild.

Yeah, it's great. It's a pleasure to be here with you, Joe. It's been a while. In fact, I think it was almost one or two years that I have been here in the show.

Yeah, you're right. I think it was 2019. So for the folks that missed that episode, is there anything in your bio that I missed that you want the Guild to know more about?

I think it's. Yeah, basically. I mean, more than 15 years in the load testing side.

And the last 10 years since. Since I've been working with Neotis, I've been heavily investing myself in building plugins and protocols. Any Toolkits or any solution that will be meaningful for practitioners. At the end of the day, we all want to validate.

How the application behaves under load and the performance side. So sometimes you need some small toolings to, to make your life easier. So that this is what I'm trying to build for the community.

Absolutely. So Henrik, I want to catch up a little bit. Like you said, it's been a while since we last spoke, so I love speaking to vendors because I know you speak to a lot of different companies with a lot of different needs. So you have a certain pulse on the industry that I think a lot of people don't see when they're just working in their own little piece of the world. So any trends or things you think performance engineers need to know a little bit more about that you see growing in the next couple of years?

I think there's clearly again two worlds that is combining at the moment. They are the classical enterprise applications that we have a lot of our customers and partners are dealing with. So those will be more like package apps. Like you don't code, you customize or you, you change the configurations and then you measure how it goes. There's tons of them on the market. So there are obviously a lot of people focus on how can I bring value in a short time under obviously a lot of stress because those products usually cost quite significantly a lot for the organization. So there are large expectations about this. So this is more like, I would say the one shot big bang load tests, they are still happening. Sometimes I think we can do a better way in that specific package apps environment. But then there is the, I would say the modern world that is popping up. So all those companies that has invested in technology that are compatible with the cloud, so microservices where they try or they have started at least to implement early performance engineering in their day to day building process, they are trying to modernize the approach. And I think now the next step is more real ability. So you mentioned sre. I think load testing is obviously part of the SRE picture, but again you don't test the same thing from a, from pure dev perspective and a pure SRE perspective. I think that the next, the next step here is more bringing value for SREs in the load testing. Combining chaos engineering with load testing and so many people around us. I mean there's so many companies, so many software vendors, they're coming up with AI. So is AI buzzword? I don't. It could be, but of course AI is a technology and in many different Areas in the quality engineering space, AI.

Could help you a lot, especially now these days where we have so many things to do. So having everything done manually could be very problematic for organizations. So we have to be smart and maybe machine learning and AI is the right stuff for us.

So you did mention sre. And what can vendors actually do to help make the performance data we all have to deal with more accessible to other people on our team, like sres.

I think sres are more close to the production side and upsides. Obviously the first thing similar to performance engineering there is a real need of understanding, measure, observe or closer to production. So they obviously wants to measure the risks, automate things to secure productions and reduce any outage, potential outage or failures. So I think that the first thing that you want to implement here is similar to performance engineering is you need to understand your system. So observing, so putting solutions like.

White box monitoring, like EPM product of course is the biggest milestone. And then you need of course to collect data from your apps. So telemetry is obviously the right hot technology to do that. And once you have a pretty good understanding, then you have the right.

At least metrics and alerts set in place. So then you can measure the risks and implement of course.

Chaos engineering, but also load testing. Because chaos engineering without load it could be quite weird. But again, load you need load testing. And performance testing is like I said, is always a simulation. So if you never do the right simulation that reflects your real life, then of course that won't bring any value for you. So understanding and know your system to be able to run the right tests, this is crucial. And I think with all the modern APM vendors, all this technology, telemetry.

It'S become easier. And I think the big hot thing with the SRE is of course the notion of sli, the notion of SLO and sla.

Where I think the biggest question that teams could have is which indicator should I pick? Which if I'm referring to chaos engineering, which type of attack or disaster I want to simulate during my test and those of course you will be able to know which indicators or which tag you have to imply in your tests by knowing and by observing your production. And this is now with all those modern APM solutions, I think we now have the keys to implement this mythology within our organizations.

So you also mentioned AI. So as a vendor, how do you see AI playing a role with performance testing specifically?

I mean it's been three years internally at Neotis that we've been working on prototypes, implementing. I would Say more the machine learning side.

Because I think the biggest pain that we are facing here in the load testing. So because neoitis is load testing vendor, so we are trying to figure out where we can implement in a smart way AI and machine learning. And so we looked at the various areas that could be worth so without naming them, of course we have to design so how could we make the design faster? And I need to analyze, so how can I detect things automatically? And of course last is I need to make sure that my tests are relevant. And so we, we decided to pick the topic that would be more helpful for our customers and prospects and partners which is the how can I scope and identify precisely what should I test based on observations. So we've been working, so now we have, I would say first release of this solution that we are implementing at a few customers. So it's a solution that attract the traffic that you have in your production environment and machine learning will be there to basically detect common user flows, common user actions, showing you what type of test case you should test in your load testing, what are the think time, the pacing, what is the load policy that you apply and and so on and so forth. So this is what we've been working on. I think for now we have a pretty good solution. We are eager to show it more globally to the market, but we are just waiting to get enough maturity from, with the help of feedback from partners and customers.

To sort of share it to the market like this. I think that will be the first milestone. Like I said.

The biggest thing that I see around us is that it happens a lot, quite frequent, that tests are not reflecting in real life and this is causing a lot of bad reputation for the industry. So I think if we are able to save that with the help of machine learning and AI, I think yeah, we will succeed. And then we can maybe think of where we could apply machine learning in other areas of the project, maybe analyze this or design a convention. So let's first deliver and save that pain of building the right scenarios and then we figure out what would be the next milestones.

Absolutely. And I think one of the issues with performance testing as well is you have a lot of people that don't necessarily understand it, but they get a tool and they overdo it. So I think there's a big need for education. So along those lines, I think you've been doing a online event for I think over five years now. I might be wrong, maybe it's longer called pac. So for the folks that aren't aware what is the PAC or the Performance Advisory Council about?

The PAC has been created initially to help the community. Like you mentioned, the problem with performance and load testing industry is that there is no clear education.

And so people have to learn by frustration, failures and projects. And I think there are so many people doing online awesome things around us. So why don't we ask those experts to come present share. The idea is to help the community to learn and understand what are the new techniques, what are the new frameworks that maybe they could look at it. So I think that was the main purpose of the pack. So it's a free conference where we try to gets the best experts of the industry and then they are here to present a topic and share it to the community. So we have two, we've built two distinct format of the pac, the physical PAC and the virtual. Now the physical PAC this year because of the current Covid crisis, we thought that let's reschedule it later. Maybe we will have a chance to organize a real physical event again. So that's why we edit the virtual event on March this year again with a pack hero. So the difference between the two events is that the physical one is like it's not an open event. We sort of come in a nice environment, we bring 15 to 16 experts and we talk. It's similar to to the things that has been built a few years back with wapor. So we try to get some topics, we discuss, we share and then we exchange with the various experts. The other format is the virtual one and the virtual is more accessible for everyone. So it's a 24 hours live stream conference. So the concept is very simple. We start.

With New Zealand and then we follow the sun until we reach California. So it's a 24 hours marathon for me at least as a moderator. And yeah, during 24 hours you will have the chance to listen to one of the 20 speakers that will be part of the conference.

Nice. So I think every year has a theme and this year what's this year's theme?

So this year's theme was. It was rock and roll. So we're going to pretend that we are the organizer of a rock festival. So every speaker will be a rock star. And so there will be various stages during our festival. 24 hours festival. Everyone will be able to come and listen to their great performance songs that they they have to share with our community.

Since you have six performance rock stars. But it looks like there's eight sessions. How many sessions are there going to be? I assume more since you're going 24 hours.

No, there's obviously more. We wanted to start promoting.

Six by six. So we started last Friday with sharing the name of six of the speakers. This year we wanted to do it a bit different because I think we covered quite well the various topics related to performance, load testing, how to automate all the pipeline approach and so on over the last few years. And I think this year I wanted to have of course themes related to performance and load testing as usual, but also bring experts that could give recommendation more on the code level. So both worlds, performance on the code, performance on the app. So we will cover both things. We will have. I'm pretty excited this year because we have pretty good speakers.

In this event. So we have a couple of speakers from Facebook. So Goranka, she will talk about capacity planning for private cloud at Facebook. We will have Denis Barkov who would explain how to become good at optimizing your code. We have a couple of also.

A great engineer from Facebook who basically did a recently performance summit Taras.

So he will be also part of us, part of the experts. So. So there will be a mix between traditional load and performance testing and pure things related to the code. I think that would be very interesting. So both worlds will be covered during this specific pack Hero.

Nice. So why the emphasis this year on having a more code level type tracks? And when you mean code level, what does that mean? Is writing performance testing code or actually implementing more performance more on the app level?

Not on building your test cases but more on a couple of concepts about when you build your app, things that you should consider.

Basically the best practice around that. So I think this has never been very covered over the last pac. We've been very focused on performance and load and I think it makes sense to open doors as well to more speakers that could also address the developer community as well. So the because now.

It'S what everyone is facing as of now is performance engineer is definitely in direct relation with developers. So it makes sense to cover topics for traditional performance engineer but also developer that also do performance activities as well.

So it's a shift that's been going on shift left. So as more developers get involved making their code more performant, it makes sense because rather than waiting until last last minute as we test it from a testing you can't test anything that's not testable. So I guess you can't put in performance that's not performant. So it sounds like more how developers can create more code that behaves better performance wise is what it Sounds like.

Yeah. So I'm very, very excited to do that. So there will be the big news I would say on the format of the pac. And also one, one big thing about this PAC from my perspective because it's, it's for me, I will be, it will be my last PAC as a moderator. So for me it's like the big epic. I try to, I want to make it epic and.

Fun again because then. Yeah, because I won't be the next moderator for the next pack. So I'm eager to have awesome last PAC from a moderator perspective.

Interesting. So do you have a new moderator lined up? Is it because this is something that you're going to phase out or what's going on there?

I will announce it on the day of the pac. Let's keep.

Cool.

All right. So you definitely have to stay tuned, join the event and as you mentioned, it is a free event. All people need to do, I assume is go to niotis.com do you have an easy to remember link that people can go to to register?

It's a very complicated LinkedIn URL but we have a LinkedIn page so if you search for Performance Advisory Council you will find a specific LinkedIn page there. Otherwise the website is www.neotis.com so n e O-T-Y-S.com performance.

Advisory cancel.

So you did mention this is going to be kind of cool because you're also introducing code level type topics. Are there any other more like red meat for performance engineers as well sessions that you're interested in. You think people are going to really be blown away by maybe. Scott Moore I think is joining Steven Townsend, Andres Grabner, people that I think performance engineers know already.

So there will be of course yes, Scott Moore will be there. Mark from Tomlinson from Perfight will be also there. We also have our friends from Akamas that will cover the optimization aspect. There is also Lloyd Watts so he's been quite active on the social media and I think his platform is interesting So I want to make sure that everyone understand the value of his platform so measuring the last mile through real devices. So I think it, that's why I think it makes sense that he comes and explains comparing traditional cloud load generators and and the value of his own platform. So I think that will be beneficial for everyone and we will couple have also Arinder Sira so based in Melbourne, he will share his experience with DynamoDB. So I think that will be also interesting. And of course we have stained shepherds that will be there focus more on the database site. So that will be interesting to to have his own vision on this aspect as well.

So I think it's a point. Even though you are a vendor, this event is very non vendor centric from my experience. Is that correct?

Yeah, we don't care of tools. I think you mentioned people are when they do performance, they try to assimilate performance to a tool. Tools are tools. I mean neotis do a great job with neoloda but here.

The main objective of the conference is not promoting our product or any other product. It's more about sharing mythology tips or even if there is any new framework that could be beneficial for anyone. So even if it's sponsored and powered by a vendor like neotis, we are not here to promote any tools. So if you want to talk about things that you've achieved with, I don't know, JMeter or K6 or anything that are in the market, feel free to come and talk to this conference.

So I know this was about the festival itself. I also just want to touch base with NIOTIS because there are people that listen that do use niotis and you're always working on new releases. I think you have seven released anything that you want to announce with the new release of NeoLoad or anything that you think folks may want to know about.

So a few things. So yeah, we're doing a release every eight weeks. We've been the last few years we've been investing and putting a lot of effort to build our web technology. So NEOD Web, it's been there now since two years but we are adding more and more features on this. So the solution to talk about that scans your production and drive your test will be coming soon. On this we are adding more and more design aspects on the web, more analysis. We have a lot of API now. So for people who are looking for collecting the raw data and feed it in other system we have API for this and we also have worked on an integration with Gremlin. So Chaos Engineering. So if you have to do any load test and simulate or any failures or disruptions on your environment. Now with this technology you can drive your attacks directly near load so and you'll be notified in your test. So it will be very helpful when you have to combine both both activities. So usually Chaos Engineering is on one side, you have load testing on the other side, but there is nothing communicating between the other. So we've been working on this. We also been working on a lot of integration with test management platforms because I think it's it's me makes a lot of sense for for any projects to have automated reporting. So if I run a load test I want to make sure that the project leaders or the team leaders they are aware of what was the results where they can find the status of my test. So this is what we've been working quite significantly and over the major leagues that we have is this summer. We have a lot of great things coming more on analysis sites. So yeah, stay tuned, you will learn hear a lot of great things about new features over our platform.

Okay Henrik, before we go, is there one piece of actionable advice you can give to someone to help them with their overall performance engineering performance testing efforts and what's the best way once again to find contact you or learn more about PAC the Pac Hero Festival.

So I would say if you do load testing the or you have to test your performance deprecation of your application, I would say yeah, first of all try to understand your app. Don't start testing if you don't know your app.

You cannot test the black box. Now you have to understand what's in your box, how the box is behaving and then you will be able to deliver the right tests and the right way of analyzing and collecting data to measure the level of performance. That will be my major advice on the pack. If you want to reach out to us, I think the best will be LinkedIn if you if you have a LinkedIn account, we are trying to be active on this or you can send an email directly to me. So Henrik H E n r I k.rexhead r e x e d@neotis.com thank.

You Henrik for your performance testing awesomeness. The links to everything of value we covered in this episode. Head on over to testguild.com P60 and while you're there, make sure to click on the Try Them Bolt Today link under the Exclusive Sponsor section to learn all about SmartBear's two awesome performance testing tools, Load Ninja and Load UI Pro. So that's it for this episode of the Test Guild Performance and Site Reliability Podcast. I'm Joe and my mission is to help you succeed with creating full Stack automation awesomeness including performance and site reliability testing. As always, test everything and keep the good. Cheers.

Thanks for listening to the Test Guild Performance and Site Reliability Podcast. Head on over to testguild.com for full show notes, amazing blog articles and online testing conferences. Don't forget to subscribe to the Guild to continue your testing journey.