Keeping Healthy and Cost-effective Kubernetes with Amir Banet

Get ready to discover some of the most actionable DevOps techniques and tooling, including performance and reliability with some of the world's smartest engineers. Hey, I'm Joe Colantonio, host of the DevOps Toolchain podcast and my goal is to help you create DevOps toolchain awesomeness. Hey, it's Joe. And welcome to another episode of the Test Skill DevOps Toolchain podcast. Today we've talked with Amir Bennett all about keeping healthy and cost effective Kubernetes. Amir has more than 20 years of software experience in software development and product management. These led teams in creating solutions to streamline the work of developers, DevOps QA and IT professionals at companies like Hewlett Packard Enterprises, C Light, SolarWinds and a bunch more. And he's now leading Perfect Scale to become the industry standard in regards to K8 optimizations and governance. Really excited to have him on the show. I've known him for a while and you don't want to miss it. Stick around. This episode is brought to you by SmartBear. As businesses demand more and more from software, the jobs of development teams gets harder and harder. They're expected to deliver fast and flawlessly, but too often they're missing the vital context to connect each step of the process. That's how SmartBear helps. Wherever you are in the testing lifecycle, they have a tool to give you a new level of of visibility and automation so you can unleash your development team's full awesomeness. They offer free trials for all the tools, no credit cards are required, and they even back it up with a responsive, award winning support team. Show on your path to great software. Learn more@smartbear.com today. Hey Amir, welcome back to the Guild.

Hey Joe, thank you for having me.

Great to have you. So I guess before we get into it, I usually botch bios. So is there anything in your bio that I missed that you want the Guild to know more about?

No, I think you covered the vast majority of it and yeah, that's fine.

Awesome. So, you know, just thinking of your buddy bio, I know you've covered pretty much every stage of the DevOps pipeline or software development quality testing. So why focus in on Kubernetes now when I think you could probably have your choice of different gigs to get involved with? Yeah.

So first, I love learning new areas. So this is, you know, the experience of going to a new area and mastering it. It's always something refreshing that I recommend anyone to do from time to time. And when we first Met I think I was the product manager of QTP and U of T. This is how we became familiar with each other. So since then, yes, I've been through a few other, let's say startups or companies with touching many different aspects of the always in the IT, always providing services or solution for QA dev, DevOps, anything in between. And regarding your question, the reason now that I joined and actually started this company is first, regarding the opportunity. So we do see here a really great opportunity to disrupt a market that hasn't been addressed properly in our eyes. And second, this is my partner. So I joined forces with someone who has a vast experience in Kubernetes, knows it in the really highest level possible and he was exactly the Persona that we are addressing and he felt the problems that our solution now helps to provide on his daily kind of routine. And because he had such a vast knowledge again about the problems and the Persona and what is really needed in order to solve it in the way that will meet the needs of the DevOps or the SREs. We decided to basically to do a test run and to we started by having more than 30 different companies that we interviewed to understand whether these problems that we will talk about later on is something common. And when we understood that there are and there are no real solution that are able to address them, we had kind of the eureka moment. Yeah. We have here something that we can really turn into a successful company.

Awesome. So I know you sent me some things ahead of time to prepare. I know there are four gaps you identified that there was with current monitoring and observability tools. Are those the ones that made you think of what the problems may be that people are facing? So I want to kind of find out what those problems are. Maybe are these gaps what the problems were that made you think, okay, we need a solution to fix the gaps that we've been finding. Right.

So first I would like to say a few words about what we are doing. I think it will help to get a better understanding about later on what are the gaps that we're addressing. So perfect scale is all about day two operation in Kubernetes Done. Right. Okay. And as part of the day two, it means how you are maintaining and keeping your clusters and your application in Kubernetes both healthy and also cost effective. And there are hundreds if not thousands of different parameters in which either developers or DevOps or SRE needs to manually decide what should be the right values for them. Now, if you're not choosing the right values, it automatically Affects either the performance your end users are getting or the cost that you need to pay for these applications. And this is exactly where we wanted to come in handy and to provide data driven decision making for those Personas so they will know exactly what is the accurate values that they need to use per each one of these parameters. And when we were asked about what is our differentiator comparing solution in the monitoring space, in the observability space, etc. The main gaps that we identify are as follows. First, they are providing you data. They are not really providing you the answers or the insights that you need in order to take data driven decision in a heartbeat. And why is that? The main reason is that because they were designed to meet the needs of the pet owners and not the cattle owners. Are you familiar with this analogy, pets versus cat?

No. I may have heard it once before, but maybe you could explain it for people who haven't.

Yeah, sure. So if you have a pet or a dog or a cat, you want to know as much information about it. You want to know whether it is well fed, what is his energy level, whether it is healthy. And this is exactly what you are getting from the monitoring tools. You are getting all the information that you can think of on a specific pod, on a specific node server, etc. But this is not really the level of information that is needed for the DevOps manager or for the platform engineer or for the SRE. They prefer to have the more holistic view of what is going on. And if they need to find the needle in the haystack to be able to do that quite fast. But again, the macro is much more important than the micro. And this is exactly what our solution provides, the macro view. And if you want to easily again dive into the sick how and know how to treat it. So this was the first gap. The other three gaps are also important. It's about prioritization. Again with the monitoring tools you're kind of bombard with multiple data points, multiple alerts. How can you really prioritize what is critical for the business and what can wait? The third gap is about whether you are able to have a good alignment and collaboration between the different stakeholders. Because in the kubernetes realm we have many more chefs in the kitchen than ever before. You have additional Personas, I mentioned few of them like the DevOps, the SREs, platform engineer, it can be also finops guys and you need again to have one system to align all of them regarding what is the single place of truth and whether the action items are actually taking place. So to have governance of that and that again, not something that monitoring and observability tools are good at. And lastly, and maybe more important and very relevant only for the Kubernetes space because this is an ephemeral system that constantly changing. You need to have this decision making done as soon as possible, even in a preventive manner to some extent. Because you don't want out of memories, you don't want to have high throttling that will affect the latency that the user is experiencing. So you need again to have here something kind of predictive that helps you to take a decision in a heartbeat and even doing it autonomously on your behalf. So all of these four gaps were the reasons or these were first the pains that my co founder felt and the reason why we decided to create a solution that disrupt the market by doing things differently.

Awesome. So I want to dive into each one of those before I do. Just like I am new to Kubernetes. So just a dumb question right off the bat you talk about parameters that people struggle with setting manually.

Yeah.

And I know this. You can run it like on aws, you can run it on Google Kubernetes engine. All those parameters different per environment that you run on. So does that make sense? Do people struggle with? Okay, I need to tune it for aws, but it's not the same if I now have like a backup that now runs on Google Kubernetes engine.

So some of them are, but the vast majority isn't. So that's one of the beauties of about Kubernetes that it is agnostic to where you're running it in the vast majority of the parameters. So the same parameters. And let's give you a few examples. For example, what is the memory and the CPU request and limit this you need to define no matter which environment you're running it, it can be even your own data centers. What is the threshold to create more replicas of the same pod? And what is the nodes that you need to select? Actually the node is something a little bit more kind of relevant for each environment separately because the Amazon of the world has different nodes than the Google ones. And you have now solution that are actually taking the decision making regarding which node do you need away from the equation. Okay, so this is also something that you see the industry kind of evolving in order to reduce the complexity that is associated with the decision making that I mentioned before.

Very nice. So I know you have these four gaps, so I just want to see how you Fix them. So the decision monitoring, I would assume that were people getting overwhelmed because there's so many things you can monitor but not necessarily know. Okay, maybe I have a high level here's a problem. Let me like know that's a problem easily. But then I can drill down and have the people that in the like you mentioned a bunch of different roles and the different roles can then look at the different metrics I guess that they have to deal with. Was that the main thing?

So first is again gathering all the information and providing it in a very accessible manner. So this is I think one of the secret sauce that what we are doing is that we are helping to make the data more accessible for you in a matter that is putting focus on the insights and less about the data again the answers and less about all the data that kind of compile it. Another thing is that we are providing the evidence why our recommendation? Because we are not just again monitoring observability solution. We are not that at all. Actually we are never replacing the monitoring solution in the organization because again we are a layer above it that provides again the insights that are coming from this information. And I could give you a few examples of the evidence that we are giving in order to support our recommendation. So let's. We talked about before about the memory and CPU request and limit. So here things like the usage patterns that actually happen in your cluster and how your cluster were dealt both under pressure and also in idle mode. So we are looking at this seasonality, we are sampling every few seconds what is going on in your clusters and understanding the utilization of each one of your pods. And this is also part of the decision making that later on affect our recommendation. Besides the utilization, we are looking at many other patterns even like whether the POD was initialized during this analysis period, yes or no, how many replicas of the POD were running and whether all the replicas behave the same under the same kind of a load. So these are just few examples of the metrics that we are looking at as part of our recommendation decision making.

When someone logs in, then they'll your insights would bubble up and say hey, your usage pattern has changed over the previous two months or something. And so it automatically tells you there's kind of an issue. And then you can drill down to find out what may be causing that. Am I thinking of this correctly?

So the usage pattern doesn't necessarily say that you have an issue. Okay, usage pattern is something healthy. And this is again one of the advantages of Kubernetes that it is allowing no matter what is your, let's say, load, reload to adjust the scale accordingly. But again, in order for it to be optimized, you need to figure out what is the right values that you need to set on each parameters. So we are using the usage patterns in order to fine tune exactly the values of each parameters that you currently guesstimate on what should be the right value.

And so the fine tuning, I assume is done automatically and then it adjusts based on what it learns from what the change was.

Great question. So currently it is not automatically the autonomous fashion will be released quite soon. So this is what we are working on. You will be surprised, but when we started the company, our first belief was that we want to go to the autonomous mode from day one. And then we had a discussion with our design partners and they said, hey, hold on a second before I trust you to do this autonomous, because you need to understand by taking full control and doing it autonomously, we are basically taking control on the most important thing of the business, which is whether their application will run successfully and how much it will cost. So they told us, okay, wait, we need first to trust you and your recommendation and only then we might consider to go to the autonomous way. So we change our approach. We put a lot of emphasis on the UI to highlight why our recommendation makes sense. Our solution is SaaS. So we are collecting more than couple of hundreds of clusters as we speak from multiple companies. And we are always adjusting our recommendation or learning from the data that we are collecting and improving our recommendation engine. And now we reach to a point where our customers are telling us, okay, you prove your value, you're accurate. We want now this to be done completely autonomously. What we currently provide, and this is again available in the last half a year so so is one click remediation. So you see the recommendation, you can easily understand why it happened. You are able also to adjust how much buffer or how much we are calling it resilience level you want to set per each one of your workloads or each one of your application and based on your configuration plus our recommendation, you are able to with one click again to implement these suggestions.

So does it give you information on the suggestions then like we implemented the suggestions and here's the results now and so people know that okay, these actually worked or maybe I need to do more adjustments based on more data that I learned about the environment.

So the recommendation are for this point of time based on an analysis window that you are selecting whether it will be a month or a week or whatever, then you're getting the recommendation, you implement them and from there on to see the impact on the system. We have both KPIs and reporting that tells you again, what was the impact of your doing on the system from a cost perspective, waste perspective, and also risk. And I didn't talk a little about the risk. I think this is a very important aspect to mention as well. Unlike many other players that are only focused on the cost optimization, our focus in the same level, if not even more, is about the resilience, durability and performance issues and risk. So we're putting a spotlight on which issues do you have, how to fix them, and what is their priority severity of each one of the risks that you have in order for you again, to improve your performance, performance and to provide a better service for your end customers. So we kind of guarantee that your SLA breaches and SLO breaches will drastically drop using our solution.

So that goes into the second point. You said prioritization. That's what it does. It focuses on not necessarily just cost savings. Obviously cost savings is important, but you may have a resilience issue that's holding you back or maybe killing your SLOs. SLAs sounds like.

Yeah. So yeah, I can tell you that this is if cost is a nice to have kind of a thing. This is these days it's actually becoming more critical than ever before. But resilience and durability and again, performance, this is a must. So all organization needs to guarantee that they are providing the service and the quality of service that is expected out of them. So definitely something that is critical and one of our biggest value proposition.

All right, so I don't want to gloss over costs. I read an article, I forgot who did it is they moved to the cloud and then they noticed the charges were in like astronomically went up and they decided to bring everything back in house again. Back in the old days, we had like, you know, raised floor with their own bare metal machines.

It was Amazon.

Oh, it was Amazon. All right, so how do you get to those extremes then? Like how do you save costs and not. When companies do do this, they do it right. And it's not that extreme where, oh, we have to just pull out now and do our own data center.

Again, so few points on that. First, not all clouds are the same. Okay, so you need to don't chase the hype, but choose the solution or the technology that fits your application needs or your business needs. In the Amazon case, they chose Lambda Serverless. And in Their case for again, for the unique use cases that they had in mind for what the application should provide, Lambda was the wrong choice, so they moved back to monolith. But it's not really a monolith, it's monolith that runs on kubernetes. So but again, it's completely different kind of architecture than what they used in the beginning. And yes, you need to do the analysis in advance, don't chase the hype and make sure that you are moving doing this digital transformation to become cloud native only if it really suits your needs.

So if they used your solution, would they have noticed like based on the insights that there was an issue? Or is this because it wasn't the right fit, that even no matter what solution you're using would be able to highlight that we're going to have an issue down the road.

So our solution is focused on kubernetes and not on serverless, so it wasn't that relevant for them. But you will be able to easily see both the cost going up, you will be able to see the waste, you will be able to see the risk using our solution. So theoretically it will give them all the kind of understanding that they're heading in the wrong direction from very early ages.

Nice. So I guess the third thing you talked about had good alignment with different stakeholders, your solutions. So could you just talk a little bit more maybe how it works with different players within the team that would use it.

I can give you an example of one of our clients. I will leave his name, their name out of the equation right now. But this company has what is called development platform. Okay. So it can be built on either open source solution, it can be built also homegrown solution Backstage is one example of a platform like that. And in their case they wanted this. Some parts of our UI will be delivered directly to the development platform. And why is that? Because their development are kind of cozy in the environment that they are. They don't want to have context switching. So in their case we are building them in integration that will take again the recommendation of what to do in each one of the workload, putting it directly in the development platform. So here you have a kind of automatic integration between the DevOps that are using our UI to the developers that needs only pieces of our UI in order to take decision just on the workload that they are responsible for. This is one example. Another example can be between the FinOps guys, the guys that are more responsible for the financial observability and control to the DevOps guys so we heard multiple times that the FinOps are kind of coming to the DevOps or to the SRE with questions regarding either spike in the cost or projections about what's going to happen next quarter. Because I need to prepare my budget accordingly. And the DevOps are kind of left clueless, especially if they are asking for a drill down into a specific business unit or a specific team or a specific product. Because that's not the way that the existing kind of cost solution are working. They are providing you more on the node level or on the entire cluster level. So this is the granularity that you are getting and this doesn't fit the needs of the finops that once again focused on a business unit, focused on a product. And this is again nice use case that our solution is able to provide because you're able again to group according to the logic that you have in mind. Other whatever scope it can be cost, it can be waste, it can be even resilience issues, how many do you have? And to give this kind of over time analysis to the relevant Persona.

So it's a different dashboard. Like if I was on the Phoenix I would log in, I would see what I wanted to care about and if I was on performance I would see what I care about.

So currently because our solution is pretty new, we don't have what is called rule based access control to let's call it, to slice and dice the solution into multiple kind of ownership. So you are getting a full view of all the clusters that you have in the organization. In the very near future we will be able to again slice and dice it and change it per the rule that you entered with.

Great. So you mentioned how right now the process is Emmanuel so people can start trusting and believing what your solution is.

Telling them it's semi manual because again we are feeding them, we're giving them all they need to do, they just need to open their mouth.

Exactly. So how does perfect scale then ensure maybe the accuracy and the relevancy of its analysis?

Yeah, great question. So first this is where our machine learning algorithm takes real proud of learning more and more and the more data that we get, the more accurate the decisions are. This is also coming from the fact of who are the people that we hired in order to create those algorithms. And we have here the brightest of minds with a lot of experience in kubernetes that again brought their experience into the algorithm. And lastly this is again based on what are the metrics that we take into account in order to come up with those recommendations. So we really don't leave any stone unturned. As part of this decision making we are looking at all the factors that matter and I mentioned few already, like the usage pattern. I talked about whether or not a POD was initialized, yes or no. Looking at all the replicas of a POD is part of the decision making and many other things. Whether this was a spot node or not, I cannot tell you all the different metrics because this is part of the secret sauce. All I can say is that once you start using our solution and see the recommendation and see the supporting evidence to why we came with that, you're easily turning into a kind of FM and start acknowledging that all the recommendation does make sense.

I'm also thinking about once of the cost savings. A lot of times I think people what they think is they didn't provision enough resources for the cluster and it turns out maybe they may have over aggressive that may add to cost. So does it help you also uncover maybe over provisioning of resources?

Of course. So that's the other side of the equation. As you correctly mentioned, most developers when they define the resources they prefer to be on the safe side. Why? First, because it's not my money, so why do I care about how much I put. And second, they want the best interests of their end customers to have a good quality of service. So yeah, in most of the cases what they are doing is what is called over provisioning, giving too much resources unnecessarily. And yeah, definitely one of the things that our product is doing is pinpointing where you have the biggest waste of resources. And that's part of the prioritization that I mentioned before that is lacking in other tools and B telling you exactly how much you should reduce those resources without jeopardizing or risking your performance. So giving you again the right silver lining between those two contradicting forces from the one hand is again I want to give the best performance, but from the other end I want it in the most cost effective manner that is possible. And we do give the user ability to control again the level of headroom or buffer that he wants to set for this specific workload. So the more critical it is for the business, the more headroom theoretically the user will select.

I know a lot of things that hold some companies back from getting solutions that they need is it doesn't integrate with their other solutions and so it becomes a nightmare. So does it integrate with things like Jira or GitHub, like maybe what's on the roadmap for perfect scale to make it so it does seamlessly fit into plug and play any type of enterprise that may need this solution, right?

Great question. So we are proud of the fact that we are streamlined to whatever workflow you have in your organization. Which means that based on what is your desires that our recommendation will kind of flow into becoming the actual remediation we will kind of satisfy. Today we're supporting out of the box with integration with JIRA and with Slack we will support very soon integration with Monday Monday.com so you are able to create a ticket directly to JIRA or directly to Monday. But the real kind of advancement will be once we will support PRs. So you will be able either to create PR and then in the next deployment it will again do the adjustment or it will automatically do the adjustment and create a pr just as an FYI, just so we will know and have record about the changes that the system has done. So this is what we are currently working on and it should be available in a matter of weeks.

I know another thing that companies sometimes resist getting solutions. They don't see results right away. So could you talk a little bit? I think you have a free trial and if they have a free trial, would they be able to see the value of the solution right off the bat? I think it's a 30 day free trial. Can you talk a little bit more about that?

Yeah. So first we're as you correctly mentioned, we're providing a 30 day free trial. No commitment needed, no limitation. You can use us on home any clusters you want. And this is very helpful because if you want to do a health check just to get to know what is the health of your clusters. If you want to do the optimization even for free, again do whatever you want, we don't care. We actually encourage you to take advantage of our free Solution. After the 30 day it can be free also because we are providing a very generous community offering. So if your total cluster compute cost per month is less than $10,000 per month, you're getting our solution for free and only if it is above 10k. We are charging between 1% to 2% based on the package that you want and again the value, you are able to take full advantage of our solution for 30 days for free and maybe continue be free if you are still in low scale in your kubernetes adoption.

So Amir, you know one of the buzzwords that's going on is digital transformations and it seems to be accelerating more and more. But I see a lot of enterprises strug with it. It's going slower than they thought it should. So do you have any insights how maybe perfect scale can help with this? Or why maybe people are seeing the slowdown of bottlenecks when they are trying to make this digital transformation shift?

Yeah, there are few reasons for that. I would like to mention one that is relevant also for the value that our product provides. And this is the fact that the Persona that we are addressing, which is again the DevOps manager and or practitioner, the SRES team, the platform engineering team, they are the most, let's say pricey or critical element to achieve this digital transformation. Not because they are the ones that are getting the highest salaries. They might, but it's not necessarily the case. But because they are in some cases the bottleneck. And why? Because their skill set is so rare then having someone who has a lot of knowledge in Kubernetes is very rare. So because they are bottleneck, we want to again free their time to focus on things that are more critical to the business than just finding the right configuration or just doing the troubleshooting to what went wrong. So instead we are giving them a tool for their daily kind of a work that helps them to save hours of their time, again to focus on things that are more important for the business, like digital transformation.

That's a great point. I think a lot of companies overlook that having the right solutions in place, even though it may cost money for the solution, actually saves you more money because you're going to free up resources, like you said, to actually focus on the critical risk areas that I think you pointed out. So that's a great point for sure. Okay Mayor, before we go, is there one piece of actual advice you can give to someone to help them with their DevOps or Kubernetes efforts? And what's the best way to find contact you or learn more about perfect scale?

Yeah, great. And everything that you ask is kind of converted and can be combined together. So also for the novice users of Kubernetes, don't be alarmed from the amount of knowledge and the amount of guesstimate that you need to take. You have here a solution that can help you and guide you on how to do this second day operation. Easily done. So take advantage of our freemium solution, learn as you are kind of understanding from where our recommendation came and make an impact from day one as you are entering a new organization or as you're studying this field. And in order to take advantage of the solution, all you need to do is go to wwwerfectscale.IO and you can find how to both to read the blogs that contain a lot of technology information that you might find interesting and also start the free trial or take a demo, whatever you want.

And for links of everything of value we covered in this DevOps toolchain show, head on over to testguild.com P114 and while you're there, make sure to click on the SmartBear link and learn all about SmartBear's awesome solutions to give you the visibility you need to deliver great software. That's smartbear.com so that's it for this episode of the DevOps Toolchain Show. I'm Joe. My mission is to help you succeed with creating end to end full stack DevOps toolchain awesomeness. As always, test everything and keep the good. Cheers. Hey, thanks again for listening. If you're not already part of our awesome community of 27,000 of the smartest testers, DevOps and automation professionals in the world, we'd love to have you join the fam@test guild.com and if you're in the DevOps automation software testing space or you're a test tool provider and want to offer real world value that can improve improve the skills or solve a problem for the Guild community, I'd love to hear from you. Head on over to test guild.info and let's make it happen.