Scaling Smart: DevOps Techniques for Cloud-Native Success

Hey, if you want to learn more about how to make your apps more scalable and how to secure create secure cloud infrastructure, you're in for a treat because today we have an expert with us, Yazik, who has over 10 years of experience building and managing cloud infrastructure. He's worked for companies like Sabre and Oracle on mission critical systems, so he really knows his stuff. He's also has his share in startups. He's done a lot of different things, he's been an early joiner in many of them, and he's helped promote DevOps culture and advocated cloud native architecture from a very long time. Also, he recently co founded let's Go DevOps, which is a company that helps others design, build and maintain cloud native applications and infrastructure. He's a big fan of cloud transformation and helps others to leverage it to its full potential by choosing the right components for the job. If you want to know what some of those right components are, you're in for a treat. Don't miss it. Check it out. Hey, before we get into this episode, I want to quickly talk about the silent killer of most DevOps efforts, that is poor user experience. If your app is slow, it's worse than your typical bug. It's frustrating and in my experience and many others I talk to on this podcast, frustrated users don't last long. But since slow performance is as sudden, it's hard for standard error monitoring tools to catch. And that's why I really dig SmartBear's Insight Hub. It's an all in one observability solution that offers front end performance monitoring and distributed tracing. Your developers can easily detect, fix and prevent performance bottlenecks before it affects your users. Sounds cool, right? Don't rely anymore on frustrated user feedback. But I always say try it for yourself. Go to smartbear.com or use our special link down below and try it for free. No credit card required. And hey Jaj, welcome to the show.

Thank you Joe for the warm welcome. I'm really glad to be on the show. So yeah, let's get started.

Yeah, let's do it. So I guess the very first question that comes to my mind is why DevOps? How did you get into DevOps? Why are you so gung ho about cloud native stuff?

So well, the story starts with me finishing the college. I was thinking I will be a programmer like everyone else and back in the days the Java was a top thing that you could actually get hired. And for my college internship I was actually hired as a Java developer. I quickly learned that in the old, especially in the old days of Java, amount of scaffold that you need to do in order to build application was so big that I decided that Java is not the thing for me. I switched to C, which was a great opportunity to learn some cool stuff.

But you moved from Java to C.

As something better, as something faster. Like I was not so keen about the speed of Java in the very first editions. So like I went for C, spent a year or two trying to learn the stuff, learned that this is not my thing. But at the point I realized that I was in the really cool place where we had a really cool system and hardware components, I started getting into those meetings and starting doing the stuff with operating systems with mainframe that Sabre had at this time and decided this is way cooler from my perspective than writing code. The code can be written through JIRA ticket to somebody else, while I can actually work with the low level stuff. So I started as a system admin system engineer and when the DevOps culture clicked, I was one of the first to join in because I had a background in actually programming for a year or two, so had a little bit of this background. I was taught how to program. And for me the transition from being admin to being a DevOps was a really smooth transition, especially when I joined companies that showed me how you can use the cloud for building scalable applications. And the knowledge from the programming and system engineering was actually something that just clicked in. And this is how I became a DevOps infrastructure platform engineer. I'm not sure how to call it, to be honest.

Nice. Cool. So I guess when it comes to infrastructure then scaling, why is this so difficult? A lot of people think when you go to cloud everything just scales infinitely. You have all the resources. What is the biggest, maybe misconceptions or things that usually trip people up and they're trying to make things more scalable.

So I think there are two things. The first misconception is that cloud only claims to be infinitely scalable. There are limits, most of the people will not reach them. But we actually broke the cloud in couple of scenarios when we reached those limits. But I think this is the hard part. You probably will not have this problem when it comes to scalability. The biggest issue is with application design itself. We often get a new startup thingy and we have the system that was built designed pretty rapidly. But then we tried to launch it somewhere and scale it. And first of all we learned that you cannot have a second server because everything, including user sessions are kept in memory. So you Cannot set second server unless you are doing some kind of sticky sessions and some weird stuff. Then you come to the conclusion that okay, the application is not booting correctly when it comes to multiple servers or the shutdown actually loses some kind of data. And when it comes to scaling up and down, especially automatically, you need to make sure that your application behaves correctly. And I think when it comes to scalability, it's not about the infra, it's more about the initial design and the concepts that you are doing. If you're following 12 factor application skin, then probably your application is ready to be super scalable. Like when we are thinking about scale, we have a customer that really utilizes the cloud that for most of his month they are working on two servers and they need about 60 to 80 servers for just two or three days over the over the month. So they are launching those instances automatically, just paying for the usage for those three or four days and then they get back to a cheaper operation, something that you cannot do with a physical hardware.

All right, so it almost sounds like it needs to be built from scalability from the start. Almost like something needs to be testable. You need to build it in something to be scalable. It'd be better if they built with scalability in mind. How do you get developers then to make that mind shift so that they are thinking about scalability from maybe the beginning? Do you think they should do little POCs first before they commit to an architecture, before building a node and then worrying about scalability?

I think the scalability part comes with experience that developer have. So at some point he just writes scalable software and builds scalable systems. And for those who are starting like this is where we actually come in as let's go DevOps. Our goal is to join companies as early as possible. So like when you are thinking about the design, we can talk with you about the initial architecture that you have and say that this part, this is your database, it will not be scaling well. You should treat it as a rather hardcore component. But those parts you can actually scale up and down. So let's move the logic from database to applications. Like you would be surprised how many people are still writing a really hardcore SQL queries or use a functions in databases to do some kind of business logic. And this is the stuff that doesn't scale well. So like the sooner we are involved it's like it's the DevOps culture. So you get the infra guys and your business guys on the very first meeting and you ask them what we should Build and how and let's have a brainstorm and figure out how the application should look like in the end.

Awesome. So I start my career as a performance tester and so how do you test the scalability to see if it's performant? Do you put a load on it with the tool, how do you normally do it? Or do you just know best practices that once it does have load on it that it's going to behave correctly and scale correctly.

So yeah, when it comes to checking the application, we usually use something like K6 to generate a little bit or a lot of load to understand how application behaves. And like in many cases when you are thinking about scalability, you may test it with the numbers that you have in mind for your final product. But the bigger thing is to actually understand when you have a bottlenecks. So like disabling the auto scaling completely and checking which component breaks first and why and understanding this flow, fixing it or upgrading this part and checking what are the next things that will go down. This actually gives you quite a lot of understanding how the application will work under load and allow us to build monitoring, alerting around the application that will also do a automatic scaling and downscaling of the app itself. And this is how we start. So like when we get a black box K6 and talking to developers, understanding the flow is the key. And like for a day to day operations we put a lot of metrics in place. So like you don't need to do that much of performance tests per se, like the synthetic ones. But you can get a lot of potential issues from production or you know, from metrics and you can have theories, validate them and actually upgrade the system whatsoever.

Nice. So this may seem like a weird question. How many people know they have an issue like how hard is it to sell to say hey I have a company, let's go DevOps, you have an issue, you may not know about it. How hard is it? Like do you have to give them awareness so they know we know we have scalability issues but you actually still have to educate them. Hey, we know you have this issue that you don't know about yet. Does that make sense?

That makes sense. The question may be asked how are we reaching companies or clients as A let's go DevOps. I think that's extremely tough question because the answer is I don't know. This is the stuff where I have a lot of metrics but still couldn't figure out all of the stuff. But there are mainly two cases what I personally Love is to get in touch with early stage companies and just say that okay, we will manage your infrastructure and all the boring stuff. This means that infrastructure IT support, security, compliance, blah blah, blah blah blah. Everything that is not custom designed for your business so that the owners of the company can actually focus on building the company and building the product. Like usually in startup you have two or three of those guys. One of them is if one of them is focusing on building infra, that's like I think it's waste of his time and potential of the company so they can actually launch faster to production. If we take care of everything else, it's usually a robust setup. We can understand a lot about the business to build a platform for them. And this is one of the approaches and in here you get this learning curve of what this means to be scalable and so on as a add on to the initial services that come in to help you with initial traction. The second option that is not that fun is when somebody calls me in the middle of the night and says that I just figured out that we have a scalability issues and I usually ask so what's the case? Well, it doesn't work anymore because we hit some kind of the limit. And those are usually a second cases like where the client actually feels a huge pain of system that is not running and to us to do something with it. So we have quite a few of those companies that actually came to us with a fire on hand asking for help.

I'm actually getting a similar issue. I have a PHP worker memory limit I keep hitting. My developer has no idea. So that's probably definitely a scalability issue. That's a hot, hot mess for sure. So, so what do you do with people that have a hot mess like that? Do you just tell them hey it's too late or do you actually how hard is it to fix the issue in a way that's going to be in a timely manner? Because I guess a lot of those would be deep rooted issues, right?

So yeah, usually we just jump in, we take the best of the best of our company and try to fix it on the fly. This usually means that we are by design getting a lot of permissions to a lot of the systems. So like the NDA comes first and then we try to figure out what can we do with a issue. We were actually called twice or three times to the live production incident with you know, all the paperwork being signed in a couple of minutes. That was pretty insane from my perspective because like usually the paperwork take takes Months in here, it was minutes. And yeah, we sit until we finally figure out what's, what's wrong. So we get their team, we get our hands dirty and we start going through metrics, logs, servers, whatever necessary to actually get the issue fixed.

Nice. So you know, talking about paperwork and all that, how do you balance maybe speed and agility with all these new security compliance probably concerns that companies have?

That's a really tough question. So like, especially for the young companies, everyone needs to understand that if you have a investor and the Runway for three, six months, you need to deliver product. And usually the security is not that important. I probably shouldn't tell it on the record, but like that's the reality of young startups. Then you also have some kind of feature delivery that is in many cases more important because still you don't have a real user base or you have just a couple of users. The thing how we try to approach the security stuff is that we build mechanisms to actually control and make the stuff secure from the very beginning. So like all of our designs, infrastructure, CI, cd, pipelines and so on, they all have some kind of knobs to tickle if you want to enable security and disable it. And in many cases for the first couple of months, those are pretty low or medium set. Not to, you know, say they are low, they're medium set. And when the, when the company matures to the point that they start realizing that, okay, we have exposure, we have really valuable data, we actually have traction and we should take care of security. It's not that we are coming to the system and, you know, coming there and it's like nothing is here. We need to rebuild everything. No, no, there are those knobs already prepared there. We can just enable a couple of switches, tune it a little bit up, and we go from medium to high in matter of days usually because it's more about education than actually setting up the technical part.

And I guess that's another good reason of working with a company like yours, because you, you may not start off with security, but you've already thought about it on the back end where when it does become more of a priority, then it's ready to flip the switch then almost it sounds like.

Exactly. It's about cutting corners, but cutting them in the smart way that you don't cut the flesh too deeply. It's just a small wound for the product itself. And the fact that we actually launched a couple of startups in the past and we know how to work with them, we try to add additional of this knowledge that we gathered over the years to build it better every next time.

Now, I don't know how common this is. I know a lot of companies, I've read a few articles where they went to the cloud and then they realized how much it costs them and they're like, it costs less to go back to bare metal. Is there anything people can do to maybe help with their cloud cost to optimize when they're scaling, to make sure they're not just spending more money, make it like just going up and up and up because they don't have a good plan in place, I guess for scaling.

Oh my God, there are so many things that I can talk now. I'm a little bit afraid that I will take the rest of the show. But when it comes to going to the cloud, one thing that I usually start with is when you are thinking about the cloud the same way you are thinking about bare metal, like you pay premium for a lesser level of stability in the cloud, that's a little bit of phenomenon because even EC2, if you compare Hetzner offering with EC2, EC2 is, well, far more expensive and not that stable. But the truth is that cloud allows you to buy resources just for minutes, for the time you really, really need them. So like from the very beginning, from your design, if you are designed for those bumps that go up and down in the traffic, this means that you don't need to buy hardware that will facilitate your peaks. You can actually buy the hardware when you need it. And it starts with this simple approach, right? Then we have a lot of stuff where in the AWS you can, for example, develop really quickly and use services where you pay per request. And those are usually cheap in the very beginning, but when you reach a certain scale, they became extremely expensive. So like in the strategy, if you're doing PoC, it's good to use stuff like AWS Lambda to check out if you really have any kind of traction. But when you reach a certain level, it's far more cost wise to move it to some other stuff of computer. And I don't want you to give you a exact hint where because it can mean that you will go to ECS or to EC2 or to Kubernetes. Depends on the business and depends on the scale. But it starts with this. You have also a lot about data, how you handle it, how you move it from hot and cold storage. So plenty of options where you can upscale and downscale your infrastructure in the way it will be cost effective. And there is one story that I want to mention here. We have one customer that works with us for around 10 years. It predecesses the let's Go DevOps funding and they were focused on optimizing the cloud spend extremely. It was their very focus from the very beginning. The business had a pretty low margin, especially in the very beginning. So the cloud spend was actually quite a part of how much they were getting money. And we did a lot of cool stuff and we estimated that the infrastructure cost from the well designed architecture to what we actually managed to do by you know, doing cloud native approach from the very beginning is about 80% reduction. So they are paying like only 20% for the infrastructure compared to the full AWS without a lot of cool features like spot instances and so on. And like the AWS was still five times cheaper than what they had on their previous hosting. So from their perspective the infrastructure is actually running on pennies, not on the full price that they had with their previous hosting provider.

Nice. So I guess to add on costs, how soon do people need to worry about or do they even need to worry about like a multi cloud strategy? You know sometimes you have outages of AWS and then whole applications are brought down because they don't have like a failover. Is that something you recommend or you still think stick to a single cloud vendor and you probably that's like a one off. Once in a great while maybe it goes down.

So that's a good question because like two or three months ago we had an outage of huge portion of intranet, not necessarily aws but yeah, a lot of Internet went down. And when it comes to multi cloud strategy we had opportunity to work in fully multi cloud approach. We had a GCP and AWS being a hot hot and the traffic switch was pretty much seamless between those. But amount of work that we need to put in and amount of things that we need to resign from or like features that we need to resign from AWS and Google maybe let's try to rephrase it. If you have a really cool offering in AWS and the same kind of offering in Google usually they will have 10, 20% of really similar features, additional 80% it's something that is cloud specific more or less either in configuration or those features are just missing in the second cloud. So with multi cloud approach what you can do is you can stick only to those 20% of sorry of common stuff that you have available in both clouds. So the biggest pain point for us was actually managing firewalls because you had security groups on one hand and then you have those tags for firewalls on the other one. You cannot just merge them together. So you need to build some kind of a layer of translation. You probably are getting back to the scenario where you are doing IP ranges for different services. So you're getting back to the 80s when it comes to managing infrastructure. So if you really want to go multi cloud, it's a huge cost. So the biggest question that I ask my customers is like, before you think about multi cloud, tell me what will happen if the half of the Internet dies? Because this is what we are talking if the GCP dies or Google or AWS dies. So nothing is working. All the pages that you are visiting every day are not working. And now the customer tries to go to your page. If Google is down, probably he will not be able to search it. But like he goes to your page and sees that it's down. So will it think like, oh, the guys don't know how to build the infrastructure and the service is down? Probably not, because most of the Internet is not working. So like whatever. And this is the case of actually AWS region going down, probably half of the services for the people are not there. So in many cases it's not that big of the deal when it comes to availability. And you can actually live with the fact that you die with half of the Internet, that's okay. The biggest issue or something that I usually recommend clients is that think also about the disaster recovery. So make sure you can restore in other cloud. Probably it will not be an hour, maybe a week. But let's assume that for some reason AWS screwed up and melted their server room. I want to have a backup in secondary location, probably secondary cloud. But this is hot and really, really cold approach to just the data. So I think this is cost effective and cost wise. You should really think if you want to be online where everything else is dead and offline because that's not that easy answer.

Absolutely. But that's a good point. A lot of people probably lulled into a false sense of security. I don't know how many people still think of a backup strategy and everything, but I think that's a good point for sure. So we mentioned security already. How much security is part of DevOps for you? And does AI make it more difficult? I know there once again was an attack recently. Someone just used AI and it was able to hit a bunch of different websites just because it was able to go through all the vulnerabilities really quickly.

So when it Comes to security. I actually have a title of DevSecOps. So the security is a really important part of our job, especially if you think about young companies that don't have a CISO role and so ever. So the security part lies a little bit on us. At least we think so. And we try to influence and build it through from the very beginning. This is really cool when the CISO actually comes to the company and he's usually like, okay, I am a CISO for a couple of startups. I was CISO for a couple of startups already and I know I will be in the really deep, deep hole. And then he comes and it's like, now, you know, the first floor is actually pretty, pretty okay. The second one is half built. Now you can come in and start furnituring the place because like, we tried to build it from the very beginning. When it comes to AI at this point, I haven't saw any increased number of attacks that are driven by AI. Because if you think about it, even in the past, we have a lot of script Kittles that were downloading. There's a addition to Metasploit called Armitage that actually launches everything against the target. And you know, we saw a lot of this kind of traffic. I'm not sure if AI maybe made it a little bit more sophisticated because the army touch was a really dumb way of firing everything. With AI, you can probably ask a lot of questions to drive you into a really good direction. But the biggest thing that I'm actually afraid when it comes to AI, we are allowing more and more agents to run into the infrastructure. And I'm more frightened about the attack vector for those agents because you can actually send documents or prompts that will be targeting artificial intelligence living in your infrastructure. So you can actually make a bad actor inside of the infrastructure. And that's a little bit more scary from my perspective.

All right, so now you just freaked me out. So if they worked, if someone worked with a company like yours, then would you point that out like, hey, I noticed you're starting to use these AI agents. Just a heads up, you know, this could be a concern down the road because I don't know how people don't figure that out until it's too late. It's almost unavoidable.

So in here we come with a set of best practices, right? So if you come to me and say, okay, I want to install this agent and it will be operating on this particular set of data, if it's like, if you're Going through, I don't know, scrapping other web pages. And I ask your AI, it will go down to the agent, scrap something that's publicly on the Internet. Really, from my perspective, let it go. Maybe it will generate some cost when it comes to traffic, but that's it. We start to make those conversations where you want your agents to be hooked to a lot of data sources, especially if they contain critical for the company information or PIIs or something like this. So in many of those POC designs that we saw in many companies, it's like you put a single server and say, okay, so we had a silo for each of the application and we had a database that's connected to single application. But now let's take the AI, let's open up all databases to it and let it digest all the data to learn and to produce answer. So in here we start the discussion about what is the potential impact, how can we prevent it, what measurements we have built around the system to make sure that if the AI goes rogue, we either catch it beforehand or even be able to block it so it will not exfiltrate the data or not do any harm to any of the systems.

So you mentioned PII in my notes, have something about worldcoin and that got me thinking. Worldcoin, PI privacy. Are you concerned about digital identity and privacy from cloud? A lot of people when they hear cloud they think, oh I can't do it, I need an on prem type of solution. Do you have any thoughts around worldcoin and how maybe that raises any questions around any of these issues?

I think for worldcoin they are doing a really good job when it comes to privacy and this is one of their mission statements that the privacy is the key. And probably I cannot talk much about the details of the projects, but I can tell you that when it comes to how far the rabbit hole goes on aws, how secure you can be with compute and with data that you have, I think we reach quite far into the rabbit hole. So, you know, starting even if you think about the standard approaches, the AWS enclaves for encrypting everything, and plenty of stuff that you can pick from AWS and AWS claims and has a lot of documents showing you how compliant and secure they can be. And one of the things that often comes to my mind, we had this conversation with one of our colleagues, like if you think about a physical server room that you have in your building, if I want to get to your hard drives, I need to come to your building, take those out. If you go to AWS server room you have probably a million of drives. Finding the data there is actually a lot more complicated than you think and a lot of the concepts that AWS is building into their servers because like they even the nitro machines, the hardware components are actually making sure that you cannot mess with other VMs, you cannot mess with others data. And AWS in my opinion is doing a really, really good job when it comes to allowing you to use a security features that are extremely enterprise when it comes to design and execution.

Before we go, is there one piece of actual advice you can give to someone to help them with their DevOps efforts and what's the best way to find or contact you or learn more about. Let's go DevOps.

So I'll probably guide you to my LinkedIn so like you can ping me with a message if you want to talk about anything or you need the advice or maybe a mentor. Like if you want a mentor and you want to talk about your designs like PinkMe, we can get a virtual coffee and actually go through the issues that you might be having. So I'm happy to share my knowledge, especially with the young companies. Like I really root for the best for startups. So like if you're starting up, just come with technical questions and I'm really open to answer those.

Awesome. We'll have a link for that in the comments down below. All right, before we wrap it up, remember, frustrated users quit apps. Don't rely on bad app store reviews. Use SmartBear's Insight Hub to catch, fix and prevent performance bottlenecks and crashes from affecting your users. Go to smartbear.com or use the link down below and try for free for 14 days. No credit card required. And for links of everything of value we covered in this DevOps Toolchain show, head on over to test guild.com p195 so that's it for this episode of the DevOps Toolchain Show. I'm Joe.

Thanks.

My mission is to help you succeed, creating end to end full stack DevOps toolchain awesomeness. As always, test everything and keep the good. Cheers. Hey, thank you for tuning in. It's incredible to connect with close to 400,000 followers across all our platforms and over 40,000 email subscribers who are at the forefront of automation testing and DevOps. If you haven't yet, join our vibrant community community@testguild.com where you become part of our elite circle driving innovation in software testing and automation. And if you're a tool provider or have a service looking to empower our guild with solutions that elevate skills and tackle real world challenges. We're excited to collaborate. Visit test guild.info to explore how we can create transformative experiences together. Let's push the boundaries of of what we can achieve. O the Tescill Automation Testing Podcast with flutes and lyres, the bards began their song A tune of knowledge, a melody of code through the air it spread like wildfire through the land Guiding tester showing us secrets to behold.