EP195 Containers vs. VMs: The Security Showdown!

Security warning.

Hi there. Welcome to the Cloud Security Podcast by Google. Thanks for joining us today. Your hosts here are myself, Anton Chuvakin, a reformed analyst and senior staff in Google Cloud's Office of the CISO, and Tim Peacock, the senior product manager for risk analytics and cloud threat in Google SecOps. Tim is, of course, still taking his break, but he will be back soon. You can find and subscribe to this podcast wherever you get your podcasts, as well as at our website, cloud.google.com/podcasts. If you enjoy our content and want it delivered to you piping hot every Monday, please do hit that subscribe button. You can follow the show and argue with us and the rest of our Cloud Security Podcast listeners on our community page and, of course, on social media. So today we have a special episode because it is one of those joint episodes with another podcast we have here at Google. In this case, it's the Kubernetes Podcast. Our guest today— and by the way, notice that I'm doing a single-person banter, which may end up haunting me for years, and it would end up being quite embarrassing. Tim, please come back soon because doing banter without you is just sad. Anyhow, back to containers. So, we had this question of container security, Kubernetes security, and of course, The incident diary setup for this is, what's more secure, containers or VM? And together, 3 hosts, me and 2 hosts from the Kubernetes Podcast, would have a very lively interview in a few moments with our guest who knows everything about container security and of course security architecture as well. So without further delay, here is the episode.

Hello and welcome everyone to this special edition of the Kubernetes Podcast and the Cloud Security Podcast. We are excited today to be doing a collab podcast where we are interviewing Michelle Jubryka. Michelle, would you like to introduce yourself?

I'm a cloud security advocate at Google, a former architect for finance, places like Capital One, Bank of America. I've also worked in academia, and I'm one of the few security people who will actually touch container security and Kubernetes.

We love to hear it.

That's exciting.

Maybe not the rare part, but we love people working on container stuff. And we have a very special episode today with more hosts. We also today have Anton. Would you like to introduce yourself?

Sure. I kind of feel like I'm in this in-between guest and host based on your introduction. So, yes, we are doing this together. And I am the usual host, well, one of the 2 hosts for the Cloud Security Podcast by Google. which touched Kubernetes a couple of times. I mean, as rarely as Michelle pointed out. So, I guess that confirms the unfortunate trend of security people not enjoying Kubernetes as much. But I think that this will be very fun. And yes, I work for Google Cloud Office of the CISO. I keep forgetting about this.

And I think we have probably a 5th member more in spirit than in person, who is your co-host, Tim.

Correct. Yes, Tim is virtually here, and maybe he'll be back by the time this airs. So, this will be fun.

All right. So, on a typical episode, how much annoyance do you provide versus how much Tim provides? Or are you doing both parts' work?

Oh, no, no, no, no. It's mostly me. Tim just provides the clues and the— I can think the trendy word would be the prompt. He provides the prompt and I provide the annoyance.

All right, cool.

And for those who are usually listeners of the Cloud Security Podcast, we should introduce ourselves, Abdel.

Yes. Hi everyone. I am Abdel Sigiouar. I am a Cloud Developer Advocate at Google. I actually live in Europe and I do stuff with Kubernetes basically.

And I'm Kaslin Fields. I'm the other co-host of the Kubernetes Podcast from Google. I do Kubernetes community things. I'm a co-chair of the SIG for Contributor Experience, which we will not go into today, and also do a lot of stuff in the cloud native community. Excited to be here.

This is exciting for me because it's part of the ongoing journey that I've had with Kubernetes. I remember when I, the first time I was working for a small software company called Ellucian, and I remember somebody coming up to me, one of the distinguished engineers saying, you know, we think we wanna use this container thing on Kubernetes. You wanna help with that? I went, yeah, sure. That's the path. That's how it started.

And just for the reference, because people are mostly going to listen to this podcast on audio, when Michelle was making the face of her colleague asking the question, she was making a discussing face of like, would you, would you, would you help us?

Yeah, whatever.

Like an uninterested perk, whatever. All right, so I'm gonna go ahead and ask the first question. And the first question is approaching the question of what is more secure, VMs or containers? And I realize this is basically a big bottle of wormware opening here. So, which one is more secure?

I think that's the wrong question. I mean, it was kind of a bait.

It's a bait.

Yeah, it was bait, right? You're right.

It's bait. There's no debate about it.

And in my experience, I mean, like I said, I'm one of the few security people that, you know, I've worked at multiple places over the years as an architect, and it was always like the hot potato. None of the security folks really wanted to be the ones to partner with platform engineering or cloud engineering on Kubernetes and containers because it just seemed so fraught. The more I've done with it, though, I don't think it's a question. I think it's that standard architect response of, it depends, right? It depends on the context that you're gonna use it. It depends on what I like to call Conway's Law of Cloud Security, how your organization is constructed, and does security get along with platform engineering? Will platform engineering take on the security of containers and Kubernetes, or will they just act like it doesn't exist? So, your mileage may vary in that if you have this really good infrastructure with integrated security that is managed to support VMs, but you haven't done anything for containers or Kubernetes, then obviously, your VMs are gonna be more secure right then. But if you have a really engaged, proactive platform security team, and the security engineering folks wanna work very closely with platform engineering, you could get just as secure containers. That may not be the answer you wanna hear, but I think it depends. I think it's how you use it. I mean, how are you gonna protect the edge? Are you gonna try to expose it? I mean, it's all about the architecture, and as I called it, Conway's Law of cloud security. Those 2 main factors together are going to decide which one is going to be more secure for you.

I mean, people still debate Linux or Windows, which is more secure. I know it's a very 2004 question, but it still comes up. And I think most intelligent commentators give that answer, namely it depends on the org, right?

Yep. I remember this also brings me back to the early days of Kubernetes, very relevant for its 10th year. This was something that came up a lot in the early days and still does today. I remember a meetup that I went to in the very early days of Kubernetes where this clicked for me. The speaker gave this wonderful presentation about how differently you would have to implement security for containers versus virtual machines because they're just implemented so differently. So, it's really all about the systems that you understand. And if you understand the attack vectors and how the technology is put together, then you can secure it more well.

I also think it's a social issue. You have to have Teams that are, you know, that idea from DORA of a generative culture, right? Where people innovate together, where there's a level of trust, where they embrace novelty. I think I've been in both types of organizations. And, you know, as the security architect/engineer, I allowed myself to be schooled by the Kubernetes and container folks, by the software developers and engineers. You have to meet each other where the other person is so that you can build security together, right? When you have those kinds of teams working together like that, then you get that, you achieve Endora. You know, if you read the report, it talks about that kind of elite generative team will be that much more secure, will deliver better secure applications.

I mean, this is actually a little strange because ultimately, if it's all about the culture, then What about the real technical differences? Like, I feel like there is still a bit of a balancing question. So I want to touch on one of the more technical arguments. And I think that it's this dreaded isolation argument.

Yeah.

And some people will make a case that VMs are stronger isolation. And of course, I've seen people who very much cringe when they hear it because there are VM escapes, there are other problems. So how would you treat the isolation differences? I think that there are still built-in, not just cultural, but built-in security differences.

So, let's clarify a couple of things. I never say isolation with containers. I say segregation because it's a shared kernel, right?

Oh, you provide— you assume that containers offer no isolation?

Some people do use that word. I personally take issue with it because if it's a shared kernel, I don't consider it personally isolation. I consider it segregation. I think where that becomes okay, sure, in my experience, now, this is anecdotal, I've found more instances of container escape than VM escape. Now, I'm not, you know, a Mandiant person. I'm sure that there are, you know, very elegant, probably hidden nation-state attacks against, you know, that can generate VM escape. However, in my experience, I tend to see more container escape than VM escape. Now, I think the trick to Kubernetes, and I came up with this a while ago, is that Kubernetes is about optimization of applications that are homogeneous, right? You're trying to optimize compute of homogeneous workloads, right? So, if you're organizing, I am all about, like, you know, resource hierarchies and trust boundaries and things like that when you're designing, you know, the segmentation of a network or your infrastructure, right? So, if you're putting putting a PCI DSS application on the same cluster as some internal backend application that is, you know, just for your internal users, I mean, that's not really good, right? So, I think it's more about the guidance of how you organize things. I think that you have to be a little more careful with trust boundaries on your Kubernetes clusters. It's about between VMs and between containers on whatever. By the way, I wanna recognize that there's also Nomad still out there. A lot of people still use Nomad, right? I think it's more about how you organize your containers and your VMs, and you look at ways to equal out your mitigations, understanding that the mitigation techniques will be different in Either of those environments, right? Does that make sense what I'm saying?

I think it does, but I do have a very quick follow-up question on that. So, whenever people discuss Kubernetes security in the Kubernetes context, I think that there is usually a lot of focus on application containers, and I never hear people talking about system containers. So, the containers that make up Kubernetes itself. Your kubelet, your API server, your control plane, and all that stuff. Because that's also code that most of the time you're not building yourself. You're just downloading it pre-made, or you are even downloading a pre-made container to run your— this is assuming you're crazy enough to run your own Kubernetes, right? But like, how do you kind of factor that in, right?

Yeah. If you're self-managing a cluster, by the way, that's one thing we haven't talked about, hosted versus self-managed. I think that leads into another problem that people have. Kubernetes is a large endeavor. It's a lift, right? And it's a development lift. And I don't think a lot of people— it's almost cafeteria syndrome, right? Their eyes are bigger than their stomachs when they decide to say, no, we're gonna self-manage Kubernetes, right? And they don't understand the level of commitment. And so, what I've seen mostly from that to sort of mitigate that issue with the control plane and trying to protect it, Is you have a separate set of clusters for the control plane kind of applications like that, the management applications and stuff. And that's how you can, because then you start to segregate using the tools that you have, specialized network policies and things like that. You have lots of options to manage all of that, but usually what you do, what I've seen when people have a large Kubernetes infrastructure, is you separate that out. Right? You separate out that kind of management stuff into a separate set of clusters. You can also do the cluster-in-the-cluster thing. That's another way of doing it so that the developer has a Kubernetes cluster that's in Kubernetes, like Kubernetes in Kubernetes. You've heard of that, right?

Yeah, Kubeception.

Yeah. You can do that. I mean, there are lots of ways of, you know, attacking that problem. But the bottom line is, I think it's about, Trust boundaries. How are you gonna lay out your trust boundaries? And the problem is, it's just like cloud. People jump into cloud and they don't organize it. And then you, now you have a brownfield environment, and you have to then on the backend try to organize it. And it's the same with Kubernetes. You know, they stand up a cluster and they throw a bunch of stuff on it together, a lot of dev stuff, and then maybe somebody accidentally makes it prod. And then now you have to rebuild a bunch of clusters, and you have to build them from the perspective of segregation.

I think that's really what it comes down to is if you understand how the boundaries work, for example, in Kubernetes, you have the container boundary, you have the pod boundary, you have the namespace boundary, you have the cluster boundary. If you understand what on a technological level those mean, then you can better choose which security boundary you need for any given application. And I also want to call out that Kubernetes, especially right now, is doing a lot of work to enable new kinds of workloads. to be run on clusters, especially with the whole AI thing going on. There's a lot of work in the community going on to better enable use of different types of hardware by different types of workloads. So it has features built in for a wide variety of workloads, but how you understand the security boundaries determines how you use clusters, I think.

Which, by the way, reminds me, people who are born, or well, not necessarily born, but who use the traditional data centers, may have particular challenges in understanding these boundaries because some of them don't have direct equivalence to the 1990s that we keep joking about on our podcast. Like, your security is from the 1990s. But if your security is from the 1990s, the boundaries would not make sense to you, and you probably would make a mistake. That to me is kind of this whole cloud-native versus data center plus servers thing. It's particularly challenging for people who don't have that in their heads.

That's why—

Well, I started my career in the 2000s and worked in data centers. And I can assure you that it was still at the time, the boundaries is your firewalls and WAF for the outside world. But within the data center itself, there are basically no boundaries, no micro-segmentation, nothing. Pretty much everything can talk to everything. And that's how, I mean, like it was just still true 15 years ago. So I don't think it changed as much.

Yeah, I think this is where identity actually becomes really important. And that means application identity, because you're not, you can't think about it like the IP addresses are fungible, the IP addresses are ephemeral, right? It's not this immutable environment where you have an IP address and you have a network. It's now just temporary. Everything's really, it's the same with certificates, and that's why it's so important for you to have identity in terms of, you know, TLS and mutual TLS, and understanding that that's the way you're gonna achieve security now, or one element of it, right? And traditional environments, it messes with, and I go back to Conway's Law, because if you have this environment that's very dependent, the security is very dependent on these traditional segmentation tools and methods, then you're gonna drop in Kubernetes. And if you don't have members of a team that are willing to learn that, that are willing to, rethink the way they're segmenting and putting in controls, then the security is not gonna happen, right? It's lost. Service mesh, that was the term I was looking for. I don't know why it left my head, right? And everybody thinks service mesh also is gonna solve the problem. And that's, I've seen that a lot with network security teams. They go, yeah, we're just, we want some of that service mesh. What they don't understand is they just signed on for a level of complexity that's about to blow their minds, right? And yeah.

We've talked a lot here about how, how you implement your security depends a lot on understanding the underlying systems. And one thing I want to discuss with regard to security is attack surfaces. That's a very broad concept, I think, in the security world. And I think a lot of folks out there, a lot of the problems, I think, come from folks not fully understanding certain parts of the system and understanding others. So, what advice would you have, Michelle, for folks considering attack surfaces of containers versus VMs?

Okay.

Well— And if I may stick my nose into this for a very quick second, people assume it's smaller for containers, which is interesting that it kind of comes as the unspoken assumption that it's smaller. And I am afraid of unspoken assumptions.

Yeah. You know what? It's the same challenges as you have with SRE. Steve Magee is gonna love me for this, right? I think, so you've taken this monolith, right? It's not as common that people are taking, you know, that they're doing these greenfield, beautiful cloud-native applications. That's what we want, right? They're so lovely and decomposed and microservices, you know, that are so identifiable. Unfortunately, that's not usually the way it plays out, right? So, you're taking these sometimes legacy application monoliths, and maybe you're deconstructing them Well, here's the problem. A lot of times, people didn't have a clear sense of what those monoliths were. Now you're decomposing them, and now they've gotta be reliable, but also secure. So, yeah, like Anton's saying, that gets really confusing, right? And so now, you've broken all this up, and they've still gotta talk to each other, and there are paths for that. And you don't know, also, you'll hear a lot of times security people will come in, and they'll say, oh, use seccomp, and all these other mitigating controls. To restrict access between the components in the right way. But a lot of times, developers didn't take the time to understand, oh, I need access to these particular libraries, and this is what my application uses. Because really, at its heart, a container is a software package. That's really what it is. And in my mind, a lot of times, the mistake is when you think it's apples and apples when it's not. Right? So, now you've got this really complexified, deconstructed application that you're gonna have to try to build controls around as it connects and communicates with all the components it needs to talk to. But you may not understand it because you're the security person. And unfortunately, the software developer and the engineering team may not have a great understanding of it because they had to— somebody told them they had to put everything in containers. Like, I've worked in places like that. Nope, we're putting everything in Kubernetes because I heard it's more efficient. So now, they're, like, rushing, and maybe they didn't— they're cutting corners, and they don't know. So, it's complicated. I think the attack surface you've had, you have all the attack surface of the supply chain now, right? You gotta worry about the container supply chain because, right, it's a software package. Are you gonna use a golden container? Do you have a process for creating a golden container? Are you gonna let your devs download it from Docker Hub, right? Do you have the security tools to actually inspect container images? Or do you have, like, this old-fashioned threat and vulnerability management pipeline that only understands VMs? Okay, so now you've got to create this whole image assurance process that you may not have. That is sort of adjacent to your software scanning process with your SAST and your DAST tools. Okay. Now, you also, though, you need runtime assurance. You need to know, okay, has there been container drift? Are people logging in, SSHing into containers, and changing it while it's live? Do you have a mechanism in place that says, if it drifts, we kill it, we bring it down, right? The attack surface is more about the processes and the lack of knowledge as you containerize your legacy stuff into that new environment. That was a lot. I'm sorry. That was probably overwhelming. I actually do a lot of talks on this. I'll give you links to— I have this great talk I used to do a couple of years ago on container security is all about the supply chain. I checked it the other day. It looks pretty, still pretty up to date. So I'll send you that.

So first of all, usually in my talks, I say that containers are just glorified zip files, but I like your description slightly better.

Not quite yours.

I may steal that.

So, and I've been to enough talks around container security, and there are typically 2 things that are mentioned very often as a factor of security, as in the arguments that makes containers more secure than VMs. Patching. And immutability. So, by how fast you can patch a container, that makes it secure. I'm putting air quotes here. And because it's immutable, it's secure. I could understand the first one. I never understood the second one.

Okay. It's only immutable if you have enforcement in place to make it immutable. If you allow Telnet and SSH or other ways to interact with a container, because once it's deployed, No touch, no touch. You don't touch it. The second somebody touches it, if you detect that, that should be killed, right? It's dead to me, that container, right? That's what it is. You have— there are security tools and mechanisms that you can use to check for that, right? That means when you say it's immutable, it's not immutable unless you treat it that way, unless you make it immutable. Like, you don't let people log into production, you don't let people— move around and interact directly with containers, right? Patching, I actually take issue with the term patching in reference to containers because you really aren't patching. Patching implies that you're adding something to a system that's running, and you're actually violating the immutability rule, right? What should happen is a redeploy. Like, if you have to update something, say, on a container image, You just redeploy it, just like software, right? You just rebuild it, rebuild the container, redeploy it out.

So careful with that though, because so we did have an episode on container security at some point, and the person we had shared a survey that according to their data, 60% of people actually SSH into containers.

And yes, yes, yes.

My reaction was, so this is like 66% of zoos have a dragon in them. Like the real number is zero, but like, why are you telling me this stuff? And that stuck in my head for 2 years, 3 years, because we all can theorize that nobody sane would ever do that. But then somebody shows up who does observability and says, 60% of people do that. So, how do you, well, combine it with the real world?

You know what? I'm gonna take this back on one. So, I'm thinking—

Oh.

Wait. So, there are 2 major use cases for Kubernetes. And I saw this, it's funny, you know, somebody, a data science team at Bank of America tried to hire me for this reason because I could understand their use case, right? Which was bizarre. But the use cases are applications, like standard, your web stack applications, right? And AI applications. And the data science kind of stuff is not the frontend stuff, but the backend stuff. The data science is a totally different use case. And I remember having a conniption fit when the data science people were like, Yeah, we're using Kubernetes. I went, what? No, no, no, that's cloud-native. How dare you? I was so offended. I was so offended by it. And then, you know, I talked to Cloudera, I talked to, like, all these different companies, and I found out how they were trying to use it. And then I realized, oh, wait a second, Kubernetes is really cool for instrumentation. Like, there are all these complicated parts for deploying data science applications, and you can use Kubernetes to do that.

Huh.

Okay, that's kind of clever. I accept that. I now accept and bless that use case, right? But that means that you separate the two, right? So, yes, you will log in. There really isn't this immutability concept with the data science use case for Kubernetes and containers. And I accept that. Are you going to mix those 2 up? Are you gonna mix the cloud-native application use case with that data science one? Remember how I said you wanna keep, like, optimized compute of homogeneous applications and how you separate them? You're not gonna, like, put those things together, right? That's really important. That's how I treated it when I would— oh, I'm gonna use it like a verb— architecting. security architecting environments. Is that helpful? Yes, people do it, and they will do it, and I will let them get away with it also if they have a dev environment, like in a strict dev, it's not stage, test, or prod. I will let developers, as they're noodling and playing around, I will let them do that in a strictly dev environment. You wanna log in? But I let them know, by the way, you know, I don't try, I'm not one of those security jerks, that tries to implement the exact same security levels and controls across dev, test, and stage, right? And prod.

So, which, by the way, reminds us of another thing which you alluded to, but maybe it's worth a very quick visit. How about the argument that people misconfigure containers more? And that may come from the fact that they don't understand them. Well, because they SSH into them, I guess that's a good argument. So, is there anything inherent in the system we're discussing that makes it more prone to misconfigurations? Like, it's a layer of complexity after all. There's an orchestration layer. So, what's your take on this misconfiguration proneness of containers?

I don't think that's— I think it's more a case of—

Don't be an idiot?

No. I think it's more a case of not having the right tools deployed sometimes to check for immutability and drift. I think, With VMs, you have Ansible and tools like that that can go in and harden, and they interact well at the VM level. We don't always have the same kind of tooling available for containers when we're deploying them. And honestly, I think there's also a sense of, I used to make a joke about Kubernetes that it was a developer response. It's actually a social response to how the infrastructure folks took over the cloud from the devs who went, Yeah, we'll teach you devs, you infrastructure folks, not to give us a VM quickly, to make us wait for machines for, you know, 3 months. We developed this new thing called Kubernetes, and we never have to talk to you folks again. And then now, the security people and the infrastructure people, they got wise, and they went, wait a second. So, I think what happened is a lot of the security and governance processes In a lot of organizations, they just haven't caught up. You can do the same controls. It just, maybe you need to get some specific tools. You need to change some processes. I think you can misconfigure both equally. Your processes just have to catch up.

I think I'll just add one more thing to that. I would be curious about a survey that asks people, how many of you are not pinning your dependencies inside the container to a particular version and just using latest of whatever library? Which means every time you're rebuilding your container, your code is changing and your container is drifting, right? I would be willing to bet it's a lot of people.

If you're not, you know, hopefully you're using something in your pipeline that you're using SCA and software composition analysis, and you're using, you have some kind of container-aware security tool that's scanning it and telling you that information. What I usually do, by the way, is my pipeline, I'll typically have a software application security pipeline. I'll have an adjacent container security pipeline. And the things that you're doing, first of all, you're plugging in and you're scanning all your containers that are in your repository. You're checking as it goes to deploy to prod or wherever, test and then prod, you're checking, hey, have I seen this container before? Has the checksum changed? Oh, the checksum changed. No, no, no. Go back, get a scan, and the scan's gonna kick off because it doesn't know it. Now you're okay, your CVSS is 4 or 5. My rule says you can't be above 8 in this environment. Oh, wait, no, you're going to prod. You can't have anything. You need to be below...

0.

3 or something, you can go to prod, right? So, you've gotta have— I have something called DevSecOps Decisioning Principles, which I created. And security is really more about clearly establishing your policies and your trust boundaries and what those policies align with. Like, I will allow this in prod, I will disallow this. This is what is acceptable in stage or test. This is what is acceptable in dev. You establish that, then the material, as it goes through the pipeline to any of those environments, it either passes or fails those sets of rules.

Yes.

So, now, that way, it's the same, and there are less arguments because everybody knows what the rules of the road are. I also really am a huge fan of release management and having really tight release management rules and having them automated as well. Same thing. And containers just fit into that, as do, by the way, VMs. I treat VMs the same.

I think a theme throughout our whole conversation here has been that I think for a lot of us in our careers, containers have been kind of the new thing that's been up and coming. Of course, now containers are more than 10 years old. Kubernetes is 10 years old, and VMs haven't gone anywhere. When discussion in the early days up to today, been, are containers going to replace VMs? And I think we've proven over time that containers and VMs are going to coexist for the foreseeable future. So let's talk about that future. What do you see in the future of both container and VM security?

Wasm.

Really?

I'm going to give you a different answer. I am really excited about the potential of Wasm. I have not personally spent a lot of time with it. You know, I'm a dilettante with Wasm, just like I am— it's the same thing. It's almost— Wasm is kind of in the same place as service mesh. I know there are a lot of places, a lot of huge places that use service mesh and use it very effectively. But a lot of times in the enterprise, everybody says, yeah, we're gonna get some of that. And then it just hangs, like, nobody ever gets very far with it, or they use 1/10 of its capabilities. Wasm is kind of in Somewhat similar place in that a lot of people got really excited about creating this, I don't want, it's not a container, an artifact that is slimmer, that's really, really slim in performance. So, you have a much smaller attack surface. Why? It's like distroless and from-scratch containers on steroids, right? Almost. Because you've got this, like, really slim, artifact that can run, and Kubernetes now understands it and can deploy it. I don't know, honestly, you'd probably know better than I how many people are leveraging Wasm, or if it's still kind of like, will it be the next IPv6, right?

Yeah.

Everybody says we should have, and, you know, everybody's still doing, you know, NAT and PAT with, you know, they're still doing translation at the edge with IPv6. Is it gonna be the next big thing? I would love to see it be the next big thing. It seems promising. I know there are a bunch of people out there who have jumped on it. I think that's what I'm excited about in terms of Kubernetes security.

I like your comparison to IPv6. It does feel a little that way.

So it makes me so mad because, you know, IPv6 should be here, but nobody wants to change.

But do you think the future is us SSHing into containers in 5 years or not?

No.

No. Good. Thank you for your optimism.

Yeah.

I mean, it's honestly, some people shouldn't be SSHing into anything.

Right.

And I mean, like, really, you want immutability. Cloud is about immutability, right? And you want, like, I worked at one place where we had this rehydration thing. And Containers should have the same. Like, I think for rehydration, I think it was 30 days. And a container, regardless of whether or not it has any vulnerabilities in it, or whether or not it's set off a drift alert, some places will proactively just, nope, we're gonna expire them after a week. Let's replace it just to be safe. Because you don't know, sometimes you have zero days, you have— Your alerting systems may not pick up on anything. So, that makes sense. Go ahead and proactively replace it. Now it's really immutable. Now you're looking at ephemerality and immutability perfection, right? Not everybody's there, obviously, but—

I'm gonna just plug a shameless plug here. I have a talk on WASM. I'm gonna link on the show notes. So, if people want to understand WASM, they can.

Yes, go see that talk. I'm excited for that, 'cause I'd like to do more with it. I just haven't had the opportunity.

And if you're not familiar with WASM out there, it stands for WebAssembly. Originally, the concept was for browsers. It was about allowing you to use various different types of programming languages across web browsers instead of just using JavaScript by doing kind of, uh, to me it feels kind of like a JVM-style compilation of, of code into this WebAssembly concept.

Blasphemy!

Really?

Did you? I, in my head, I'm comparing it to JVM too, but But, you know, we don't want to go there. Yeah, let's not go there.

And the infrastructure people somehow saw this and decided this might be a good approach for making process isolation container-ish concepts even lighter weight. So, it's actually a very interesting technology. I would recommend checking out a talk on it, like Abdel's.

I'm excited to see that because I'd like to know I think people are using Wasm potentially to minimize the sidecar stuff. That's what I had heard, you know, as much as I try to keep up with it. And I think that's great because a lot of times in Kubernetes, you know, your sidecars and your system, the Kubernetes control plane stuff itself is gonna be just as vulnerable sometimes as the, you know, application people's containers, which is...

And to wrap us up here, I wanted to ask you, Michelle, about another talk that I am excited to see, your session at KubeCon. Would you tell us a little bit about it? Sure.

Basically, it's sort of a career talk about, I call it, it's called Why Perfect Compliance Is the Enemy of Good Kubernetes Security. And I work on the CIS GKE benchmarks with the GKE team at Google, in Google Cloud. I've been a consumer of the benchmarks, and now I've been a contributor. And the one thing, over and over again, I end up being the mediator in the middle, usually between the platform engineering team and the security team. I don't know how I ended up in that role, but I'm usually the one that's like, hey, let's calm down and talk about what's realistic here. Do we really need this specific control? Does this apply here? What's the trust boundary? What's... What's the attack surface like? What's the exposure? What's the risk involved? And I think that's the point of my talk. I wanna create a more— and this is from experience. I remember walking in and I took my happy little CIS benchmarks, and I remember working with my friend Subhendu at Bank of America. He's great, you should look him up. He works on OpenShift, right? The Kubernetes we don't like to talk about, I guess. I don't know. I would say, hey, turn on this because the benchmark said to. I kind of understand it, but, you know, and I turned it on on Minikube, but I don't know, or Minishift, and I'm not sure how this works. He's like, Michelle, that's an alpha feature. I don't think it's a good idea to turn that on right now. And those are the conversations that we should be having, right? We should be having, and that's what the talk is about. Let's have better conversations about, hey, here's this feature, what's it really mitigating? What risk are we trying to address? Let's be conscious of that when we plan out and when we try to secure it. It's not about turning the security tool to a unified green color, right? Stop freaking out about the red and yellow in your security dashboard, because sometimes it doesn't understand context. I think that's the TL;DR for my talk at KubeCon.

Wonderful. So, if you're attending KubeCon North America in Salt Lake City on November 12th to 15th, something like that, then make sure you check out Michelle's talk if you're interested in learning a bit about how to have those important conversations when you're developing a new feature. about how you're going to secure them. Thank you so much, everyone, for being on today for this very special episode.

Thank you, Anton. Thank you, Michelle.

Thank you. That was fun.

Yeah, thank you.

And now we are at time. Thank you very much for listening, and of course, for subscribing. You can find this podcast at Google Podcasts, Apple Podcasts, Spotify, or wherever else you get your podcasts. Also, you can find us at our website, cloud.withgoogle.com/cloudsecurity/podcast. Please subscribe so that you don't miss episodes. You can follow us on Twitter, twitter.com/cloudsecpodcast. Your hosts are also on Twitter at Anton_Chuvaikin and _TimPico. Tweet at us, email us, argue with us, and if you like or hate what we hear, we can invite you to the next episode. See you on the next Cloud Security Podcast episode.