5 Top Factors For a Fast, Performant Database with Peter Corless

Foreign.

Get ready to discover some of the most actionable performance engineering testing insight reliability device with some of the world's smartest engineers. Hey, I'm Joe Calantonio, host of the Test Guild and SRE podcast and my goal is to help you succeed in creating application performance awesomeness.

Hey, it's Joe and welcome to another episode of the Test Guild Performance and Site Reliability Podcast. And today we'll be talking with Peter Carless all about the best five factors when selecting a high performance low latency database and probably other topics like performance and testing related topics relating to databases, which I think a lot of people may overlook if you don't know. Peter is the Director of the Technical Advocacy at Ciliadb and he is the Program Chair of the annual P99 conference and the CILIADB summit and he hosts CILIADB's masterclass event series. It's actually where I found him. I think they did a masterclass with K6 and Dynatrace and I thought this is interesting. So that's why I wanted him have him on the show because I think this is a highly overlooked area, especially when it comes to performance and performance related issues with your testing. So you don't want to miss this episode. Check it out.

This episode is brought to you by the awesome folks at SmartBear. Listen, we know load testing is tough but necessary. So investing in the right tools to automate tests, identify bottlenec, resolve issues fast saves your organization both time and money. And that's why SmartBear created Load Ninja, a SaaS load testing tool to help teams get full visibility into performance so you can release quality software faster than ever. Make testing effortless and give it a shot. It's free and easy to try. Head on over to loadninja.com and learn more. Hey Peter, welcome to the Guild.

Thanks so very much Joe. Thanks for having me.

Awesome to have you. I guess. Before we get into it, is there anything I missed in your bio that you want the Guild to know more about?

Just one thing. We tell people that no matter how you pronounce it, that's the right way of pronouncing it right? So. But it's psyllidb. Kind of like silly but with an A at the end. So silly. B is the way we normally pronounce it. But I believe that like Homer's Greek, you would have said like Scylla, right? So I'm certainly not a professional in ancient Greek. But again, however you pronounce it is the right way for us. So perfect.

CiliadB it would sound like silly to me. I'm like, is it sillydb? So that's good to know. Awesome.

We're very silly. So it's okay.

Great. So I guess, Peter, before we get into it, as I mentioned, I think a lot of people maybe not think about the database of their application, especially when it comes to performance. So maybe to set the stage of how important is figuring out your database architecture or what database you're going to use when you're creating an application as it comes to maybe performance down the.

Road, it's pretty fundamental. It's kind of like a lot of people make important choices around, let's say their cloud provider, right? That's a key thing because depending upon what cloud provider I'm going to be at, that's going to lock me in for what, three to five years. Now you can shift from that cloud vendor, but boy, is that a heavy lift. Right? And then after that you're making other key decision, what kind of hardware platform we're going to run on. But even then that's very dependent upon what software you're going to be running on it. And so a key factor there is choosing your database infrastructure because you can swap out a database and in fact a lot of the use cases for sclydb come from a brownfield conversion over something else. But the reason why we're chosen over and over again is because people made a wrong choice somewhere along the way or they made a choice that seemed right for the MVP and then they hit some throughput or some sort of total data under management and suddenly the thing was just falling down around them and their database was pretty much not keeping up with the growth of their business. And that's the worst thing you can do because now it's a five alarm fire, you're losing business, your customers are complaining, and now you're asking me to swap out the database. So we're seeing more and more that we want to get ahead of that curve. We want to say, you know, what's the capacity you're looking to run at within a year of beginning and maybe if you're past your first year and you're looking, you're planning for years, 2, 3, 5, et cetera, you know, what is your scaling requirement? Because I think that we're seeing over and over again we can save people a lot of headaches. If you had started architecting right, with something that was going to meet those scaling challenges that you knew you're going to be facing.

So when that happens, how do they know to look at Sellia db.

Well, the thing is, is that we're not like if you take a look at, there's a site called db engines.com db-engines.com and this is kind of like the Billboard charts. If you've never seen. It's the Billboard charts of databases. Right. And db-engines.com ranks nearly 450 of our closest competitors in the industry. There's tons of databases of course, starting with Oracle and as you'd imagined, Microsoft SQL Server, et cetera. You know, those are the most popular databases and we're not even until like later in the top hundred. So we're probably not the first thing that people are going to come to. But there's certain kind of challenges at scale that people would traditionally have turned to let's say an Apache Cassandra. Right. They know about that or they might have started on all AWS stuff and they. So they easily get into DynamoDB. But the problem with Cassandra is that it has a performance lag issue, you know, just in terms of latencies and it'll certainly horizontally scale. But performance wise you can scale for total data under management, but you can't scale latency the same way. You really need to start on a faster system. And now you're taking a look at something either an all in memory kind of thing like let's say say a Redis, you know, or you're taking a look at a fast persistent store like a silly db. Again that's totally budget driven. Also budget driven is Dynamodb. Very fast sell to get started. But once you start operating at scale, that AWS bill is basically taking all your venture capital, taking that check, turning it over and co signing it to Jeff Bezos. Right. So for people who are trying to contain their spend, we're also seen as a cost effective and performant alternative.

Great. So speaking of determining what database is right for you, you did a webinar recently on the best five factors when selecting a high performance low latency database. I thought maybe we'd dive into each of those and then see where it takes us. So the first one obviously was software architecture. So you mentioned the difference between something like Redis and your solutions. What is the architecture? Why do people need to worry about what should they look for when it comes to architecture?

Sure. I mean Redis is a great system so long as let's say you're just trying to keep your data in memory and you can afford that and you're operating out of one data center. Right. Because if you want to do a multi data center deployment that's not redis go to. It's kind of an anti pattern for redis, right. Whereas for instance FacilityDB, you have multi data center capabilities built in from the get go. Right. So if you wanted to have let's say a Tokyo, Japan, an EU zone and a North America database, they're all conjoined in a single logical cluster. You can do that easily with ScylladB. It's not like a bolt on like you get with a lot of other databases. And I don't want to pick on anybody. You know, a lot of these people have really good systems to keep, you know, data centers in sync. But that's not the same thing as true multi data center clustering.

Does that help with failover? Because I know nowadays with hybrid cloud, everything's moved to the cloud. If one instance goes down, would using your help you go okay, this is down. We can easily go to the other.

Instance, true story, where there was an OVH cloud fire not long ago and they lost an entire data center in Strasbourg. Literally the whole thing burned to the ground. And we had a client who had been running there and all 10 of the nodes that they had running there were slag heap of like metal and ash, right. It was just nothing was left. And we were not the only people affected. The entire data center burned. It was like hundreds of thousands of domains were just off the air, things like that. But because of Silius multi data center topology, they also ran in two other data centers and there was a complete copy of the data at each of the three data centers. And so even with two of the three data centers running, they were able to maintain their service level agreements and totally run with nonstop operations. Like the people trying to get on a flight that morning had no problem, people trying to book flights, no problem. Everybody was able to get where they were trying to go. It was a very big travel company in Europe that had this issue. And so but fortunately again, multi data center clustering doesn't. We were able to handle it. And so there's a lot of other systems that, that would have been a complete and total outage. And there were hundreds of thousands of people caught up in that fire alone. You know, so that's an example of resiliency, true high availability. A lot of people talk ha. But then if you actually, in fact this is something that your test audience might want to consider. It's a concept I call the torpedo test, right. During World War II you would fire a torpedo at an aircraft carrier and how many torpedoes would you have to hit that aircraft carrier before it sank under the waves? Right. So when you're trying to do planning capacity planning for a cluster, or high availability planning for a cluster, let's say you set up a cluster, a test cluster of six nodes and you fire torpedo away. Boom. And one node is just taken off that cluster. Can the five handle the load? Can they handle the latency? Can they handle the throughput fire away. You know, with torpedo number two, now you're down to a four node cluster and you want to check to see when is that system no longer reliable. Like I can't actually keep pumping data into it because, you know, I might lose quorum maybe it goes nonlinear in terms of performance and at this point I just can't rely upon it to be able to handle my service level agreements. But people should know that before they go into production. And so that's again, it's just a concept called a torpedo test. And I think people, if they're doing chaos monkeying and production systems, they might want to do some torpedo testing even before they get to production to understand what that degradation envelope would look like. Does that make sense?

Yeah, absolutely. I was just going to say it sounds like chaos engineering, but obviously it's not in production. So it's similar.

Exactly. It's before. It's because people always do the benchmarking test to see what their maximum performance is. Right. Everybody needs to do that. But then they think, you know, they don't do the step back of like, so what if we're not getting peak performance? Right. And you should have that as part of like your disaster preparedness planning before, before you go to deployment. Right. Maybe do we need a 6 node cluster? Or maybe we really would be safer with seven. Do we need it for capacity? No, but we need it for resiliency, you know, so that's the kind of thing I think people need to have a little bit more proactive thought about.

Absolutely. So is there anything built into Cilia that makes testing that type of functionality easier?

Well, I mean Cilia by its default, there's a whole bunch of things that is going to make just running it easier. First of all, we're an active, active node type of topology. There's no leader replica model. And so there's a couple of advantages in that. Number one, you're bulletproof that any one node can be lost and it doesn't matter to the cluster. Right. It'll just rebalance around the existing remaining nodes. We have anti entropy mechanisms like what's known as hinted handoffs. So if you take a node down for administrative reasons, for whatever reason, and then it might be down for just an hour or two and then you put it back up, maybe you were replacing another software feature, you know, some sort of package that needed to just be updated and you didn't want to be live. While you're doing that, that update, you can do those administrative changes and the rest of the nodes in the silicluster will keep the live changes. And then when you join, it's kind of like you'll get notes passed to you like when you had to step out to the bathroom for a class. Right. Hinted handoff is a great anti entropy mechanism for transient outages. Right. And then of course it has this capability of automatically healing so that if you lose a node permanently and add a node back in that the cluster will rebalance automatically. Auto sharding. Right. So you don't have to do any of the manual sharding. These are just, I mean it's not for testing purposes. This is just how the system operates, which is like really helpful for when you're in an emergency situation. You don't need to panic. The system has these healing capabilities built into it. I think that the other kinds of things that you'll see from Scylla is also like it can do tuning based upon the workload. So maybe you're seeing a spike of work that's going on right now. Maybe for whatever reason you have the thing that everybody's looking up right now. Well, we actually have ways of doing CPU scheduling so that for instance, some of the repair or compaction mechanisms can be stepped back while transactions are heavy. Right. So it has a little bit of IO scheduling and CPU scheduling so that it will make sure that administrative tasks aren't sucking up all of the kinds of things, you know, all of the CPU usage when you really need the CPU for transactions. And the last thing we have, which is kind of nice is let's say you want to have a very transaction heavy workload, but you still want to have some analytics. Right? But a lot of times when analytics hits a transactional system, those analytics are doing like a full table scan or a range scan. And that's totally going to just gum up the works. Like what do you do when you're trying to do real time transactions and suddenly a gigantic full table scan comes in for analytics and that's for many other databases, like a panic situation, like suddenly you're going to see all those transactions brown out the SLAs are going to be blown for all of those latencies because the system's being hit by the daily analytics. So what we have is we actually have a feature called workload prioritization so that you can kind of, it's not exactly a rate limit, but you can give shares to the analytics such that they're deprioritized compared to the real time transaction. And that way you can, you know, you want to keep those SLAs on the transactions. I don't care if it takes a half an hour or two hours to run the analytics. I need my transactions to be, you know, single digit milliseconds. That's important. These are kinds of features that are built into scylladb from the jump such that it makes it easy to run it as a system. So we've heard that over and over again. I have a toddler at home, I don't need one at the office. I want my database to not be a toddler. I don't need to babysit it.

That's great. Absolutely. You know, fact one was that you talked about making it more performant was software architecture. I guess the other thing that come into play you touched on a little bit was hardware. So you know, how does hardware help? Does it help CPU memory? Does it just scale up to infinity? What's the hardware utilization that helps with the database?

So I'm going to step back. I mean both you and I have the gray hairs to prove that. We've been around this industry for a while and I remember that it was write once run everywhere was the promise of Java. Right? That was the whole thing. And that was back in an era when the operating system you're running on was indeterminate. And the kind of the legacy of Java is that it's right once run terribly everywhere because the Java virtual machines, it's like operating on a patient with waldos with like these remote systems because you don't know the hardware you're running on, you're not allowed to. You're in the virtual machine. You can't tell exactly what SSDs you have. You can't tell what kind of CPUs you're using or NUMA awareness. Like there are by the way, some newer JVMs that are, let's say numa aware, right? They are more latency sensitive. And so I'm not trying to bash Java overall, but this was always a legacy of Java is that it was one step removed from the code, from the hardware, from the metal, right? Now with a system that has access to the os, to the low level hardware, you can do some optimizations. So for instance, when Scylla first runs, we've run a tool already called Disk Explorer that takes a look at the performance characteristics of just whatever SSDs you have. We've recently run them on let's say the AWS i4i, which is a brand new system that has these really fast, what's known as Nitro SSDs. They're NVMe SSDs, the latest intel powered systems that are perfect for like databases. So we profile that we have all this logic of how fast, what's the maximum throughput versus concurrency balance? Right? You have this performance curve of how much throughput you want to drive versus how much latency you want in a database, read or write. So we've done all those analytics ahead of time and we have a utility called iotune that ships with sillydb so that when you're installing it on a particular kind of server, the I O scheduler will marry its maximum concurrent throughput. It's number of concurrent writes that you're doing or reads. It has all that kind of profile the SSD done ahead of time so that it can manage that for all of the different CPUs trying to bang on that SSD. Because while we have the equivalent of let's say memory being numa aware so that you marry that up with a particular cpu, we don't have the same kind of strict I O controller for like the CPU to a disk. Like we can't do that same kind of partitioning, right? So we need to make sure that the I O access is carefully balanced across all the CPUs and nobody's like hogging all the writes and so another CPU is starved for them. So we have that built into our system as well. But that again, you need to be able to understand what kind of hardware you're running on. Because let's say an N2 hi mem on GCP is gonna have very different SSDs than what you're seeing on the i4i on AWS. It's just very different hardware. So that's just one example, that's just the SSDs. Then of course there's all the optimizations. Are you running on let's say a Graviton 2 or Graviton 3 for AWS? Those are completely different, those are ARM based chips than an intel based chip. Right? So does your system, does it know how to get maximum performance out of any of these. And of course for Java you're always gonna see a Java based application running it, usually less than 50% of CPU utilization. It just doesn't know what to do with it. And so if you're seeing a Java database running at only 30% CPU utilization, that means that 70% of the CPU you're paying for isn't being used.

Makes sense. That's crazy. So it almost sounds, it sounds like not only does it save money though, it saves time with testing because a lot of times back in the day I would have to test against all these different configurations and then, you know, optimize and then run the test again to see if it helped. But it sounds like out of the box this kind of has almost preconfigured knows how to run on certain configurations.

Yes. So we've done as much as we possibly can to make sure, like the most well known public cloud servers that we know about, we've tested those again. Right. And then of course, you know, if you want to Disksplorer is free, you can go to GitHub and get Diskplorer and run it. Like if. So if you're running on, on premises, kind of like build your own boxes, you can at least run Disk Explorer against that. Right, Perfect.

All right, so I think we covered hardware, we covered software architecture. The third one you went over was interoperability. So how does that come into play?

It's so huge. It used to be that, let's say again, we're the gray beards here. So back in the day you would buy one database to rule them all, right, Oracle ERP and that like everything was in Oracle ERP and you had no other databases or you tried to pretend that there were no other databases at the company. Right. But these days people are making purpose built applications using maybe SQL for some workloads, they're using NoSQL for other workloads and they might even have multiple different nosqls running in the same shop. You might have, for instance, a shopping cart might be built in SillyDB, you might have a graph database and you'd have your own special flavor of graph database for like relationship awareness, things like that, or sentiment analytics on social media. Right. So there are large companies that are literally running over a hundred databases. And so you need to make sure that they're interoperable at least with the kind of a data bus that people are using across their companies. There's still a lot of ETL kind of back and forth between databases, but more and more these days we're seeing event streaming, you're seeing the Kafkas or the Red Pandas or the Pulsars of the world. And so your database needs to be able to plug into that both as a sync, as a consumer of data coming from other databases as well as being able to be a source, as a producer of data so that other systems can take advantage of it. And I'm thinking about one telecom right now that has a Scylldb instance working and they have a ton of data coming into it from an upstream Kafka topic. They then push it out sideways to a kind of a sideband conversation where it gets enriched and then brought back into Scilla with that enrichment through Kafka. And so it's a subscriber once and then a producer and then a subscriber again and then it downstream produces to get a tail end kind of reporting system. So Scylla is, it's not like just two tin cans in a string. You're having these complex conversations with your data, complex pipelines and your system has to be able to work in real time constantly with those kind of fleet based mechanisms.

Does that help if someone's trying to move over to your system from say Oracle? Because they could do it slowly with this approach.

Yeah. So there's two things that happen these days. One is that there are some people that just build greenfield or they brownfield on SQL. It's just, it's a denormalized system and it just may have to send results over to a SQL system. Or people may be doing like offload. Right. Where they're taking certain workloads that currently like we had a company that was doing gps, a lot of GPS data for real time trucking and Oracle was just an anti pattern for that because it was all transaction oriented and it couldn't keep up with the on the fly updates that you were getting from all these trucks constantly moving all the time. And like Oracle was just not the right place. So they made a no SQL system to keep track of all that telemetry data which was, you know like a time series sort of system. So Scylla was that. And then when the trip was complete, boom. You send back the results to the Oracle database. Truck rolls and then trucks complete. Like all it needed to know was the beginning and the end of it. It didn't really need to know every single step that the truck made along the way. Right. So I think that's what we're seeing a lot of, a lot of these hybrid Systems where Oracle SQL and other SQL systems and NoSQL systems, they're all staying with the same corporation. In fact, you know, now the other thing that we're seeing is people doing multi tenancy where they're sharing their databases with other companies, right? And maybe they're using a pulsar or they're using a Kafka to share out those topics consumers beyond your enterprise.

Am I right? Very nice. All right, so I guess the fourth fact then would be rasp. What's rasp?

Yeah, so rasp, it's an older term, hardware term kind of thing and it stood for reliability, availability, serviceability and performance. It can also sometimes be rendered as scalability for that s right. So but these are all the ilities, the qualities of a thing. It's not really a feature, it's not a function, it's a quality of a system. And this was used to for instance, reliability. We just talked about that case study with the. With literally a data center burning down and still operating using the availability of the two other systems. We talked about availability also and that's for instance in databases is can this database be partitioned? So let's say North America and Europe lose their connection. Can you still be doing, let's say a local quorum within each of the continents and so they're still able to operate and even though there's a partition between the two different elements of the cluster, you're still having highly available, highly performant local access and then the database will heal once that network partition is fixed. That's an important function for a lot of people that want to be running globally available kinds of systems. Let's say a mapping company wants to have again they want to have maps of Tokyo in Tokyo. Why am I serving, you know, for this mobile user, why am I serving a map of Tokyo out of London? It's just an anti pattern, right? I'm limited literally by the latency of the speed of light between here and London. So I want to give local access and if there's a network partition, I don't care the person in Tokyo, they should get map tiles served right out of Tokyo. Right. So all of this goes into that element of availability for serviceability we've been talking about just making it easier. So many databases are complex and they're frightening for developers even if you do them as a database, as a service. It's like I don't want to touch that with a ten foot pole. They want to have an API to write to and then on the back end Again, database as a service is a wonderful way to make it generally serviceable. But what happens when a database is not performing to your heart's content? Is it easy to troubleshoot? Do you have the observability tools built into this? Do you have the manageability tools built into this? And all that goes into serviceability and finally performance. Again, if you need a system that has to have single digit milliseconds, needs to scale to terabytes or beyond, needs to be able to support hundreds of thousands or millions of terabytes, there's some of these things that you can throw money at to scale, but there's other things like latency, it's got to be built to perform very simple.

So yeah, so for serviceability, developers are getting more and more on that plate that they have to deal with. And so a lot of times there used to be like a database admin, he was like the God of the company.

Exactly.

Sounds like almost this makes it so it's more dev friendly. Am I understanding that correctly as well?

Yeah, you want to make sure that the developer experience is really, it's facile to them. So for instance, just as an example, a lot of NoSQL databases decided to at the same time that they wanted to do, let's say a high availability architecture and all this goodness for cloud based services, they also decided to throw at you a completely orthogonal query language right on top of all this, make sure that everything I've ever learned about SQL is absolutely useless.

And so they go like, I don't even know how to write queries right now fortunately for like people who are used to Cassandra or used to, let's say Datastax Enterprise or Solar db, they use a thing that's SQL. Like it's called CQL Cassandra Query language. Now it doesn't support table joins, it doesn't have all the features and functions of ANSI SQL. But for a person who's got passing familiarity with SQL, it's not as intimidating, right? It's going to seem more facile, you don't have to wrap your head around it. Now I'm also going to point out though that there's a lot of people that have grown up in a world of JSON and they're like SQL, what are you even talking about?

Right?

And so for them something like the Amazon DynamoDB SDK is far more familiar because it's at least, it's basically tossing JSON at things. Right. I can wrap my head around that. So fortunately Scyllaby supports both a DynamoDB SDK point of view and it supports that more traditional CQL point of view. So however you prefer your query language, we support both.

Great. And so the last factor was deployment.

Yes. Can this run where I need it to run? Because everything I've told you before now means absolutely nothing if you cannot deploy it on my favorite cloud. Right? And now here in North America we might be more familiar with the AWS's or Azures or Google Cloud, but overseas there are any number like I talked about, OVH cloud that's big in France, there's cloud providers in Asia. And so sometimes running on a DBAAS is really convenient if that's your favorite cloud, but it could be a lockout if it's a single vendor cloud. And I just, I'm not going to be able to run that in where I'm running right now, whether in Asia or Europe etc. So you want to have something that's deployable maybe on premises these days. I know that that's a crazy thing, but there's still a lot of people running on premises gear. In fact, I don't know if you knew this, but there is an extension of AWS they call AWS outposts and it's a cage that Amazon deploys on your data center at your site and it's all AWS gear inside. And in fact it's an extension of aws, but it's running on premises and people need that for data governance or localization purposes. A whole lot of security reasons they say like, no, no, no, that's great, we don't want it in US East 1 or we don't want it in US West. We need to have this as an extension on our premises for various purposes. The crazy thing about that is that Scylladb will run on those. It'll actually run a Scylladb cloud, but running in those outposts. So it's a fully managed DBAAS on premises.

But that's an example of the flexibility that people want.

And I see as more and more we're going cloud first, people are making the transformation to the cloud. This is like the need for that is more and more critical, right?

And that's another thing too is that people want to span from on premises, you know, so they have a hybrid cloud scenario, right? So the awesome thing I just talked about with the multi data center deployment, right, you get that built in for Sally to be. It's not an extra cost thing. If you want to have some of these nodes on premises and that might be where you run your analytics is like, I'm going to have a data center that's like, I'll run my analytics locally. And the transactions are all out on AWS or Google Cloud. You can span your cluster across both of those deployments. Or maybe internally you're doing some sort of internal data enrichment, right? So you're getting feeds from internal and you don't want some of those feeds to be out on the public cloud. So. But that's the awesome thing about ScylladB is you can kind of control that and still consider it a single logical cluster.

Peter, I've got one question and maybe it's obvious, but maybe it's not. Is this open source or do you have an open source solution? Because it sounds like people want to get started. I assume if you have an open source version that'd be a great place to start as well.

So yes, you can get started in docker today with SillyDB open source and it is completely open source. So if you want to deploy that to your favorite cloud or on premises environment. Scilyd Open source is fully functional. It is not crippleware. It is just like again, an Apache Cassandra. It's completely open source software. If you want to step up, then we have Scylladb Enterprise which has got some additional features which might save you some money when you take a look at incremental compaction. We can definitely make sure that you're getting the maximum utilization out of your hardware with solidb Enterprise. And the last thing we have is if you just want to be a developer and you don't want to get your hands dirty with backups and repairs. We have Solidity Cloud which is a fully managed DBaaS. So again you can get started for free and then, you know, we'll scale and work towards your budget.

Okay, Peter, before we go, is there one piece of actual advice you can give to help someone with their database performance efforts? And what's the best way to find contact or learn more about solidb?

Right. Well, people are asking me to help them actually troubleshoot their real time production environment. Sony's gone desperate, but I would recommend that everybody go to Scylladb University. It's University Scylladb. That's S, C Y L L A D B Com. We have all of our education and training is free and online there. And then another thing I'd recommend is if people are interested in getting their feet wet, not just in Scylladb but anything in terms of big distributed systems that need to run at low latency there. We have an upcoming event called P99 conf and you can register at P99 conf.IO. it's completely free, it's online, we have nearly 7,000 registrants already. We're expecting between 8 and 10,000 registrants for the event and it's not. Again, it's going to be all sorts of databases but streaming event systems, application architectures. For anybody who's interested in this kind of career, I would say it's definitely an event you want to check out.

Thanks again for your performance Testing awesomeness Links to everything of value we covered in this episode. Head on over to testguild.com P96 and while you're there, make sure to click on the Try them both Today link under the Exclusive Sponsor section to learn all about SmartBear's two awesome performance test tool solutions, Load Ninja and Load UI Pro. And if the show has helped you in any way, why not rate and review it in itunes? Reviews really do matter in the rankings of the show and I read each and every one of them. So that's it for this episode of the Test Guild Performance and Site Reliability Podcast. I'm Joe. My mission is to help you succeed with creating end to end full stack performance testing awesomeness. As always, test everything and keep the good. Cheers. Hey, thanks again for listening. If you're not already part of our awesome community of 27,000 of the smartest testers, DevOps and automation professionals in the world, we'd love to have you join the FAM at Test Guild. And if you're in the DevOps Automation software testing space, or you're a test tool provider and want to offer real world value that can improve the skills or solve a problem for the Guild community, I'd love to hear from you. Head ON over to testguild.info and let's make it happen.