EP101 Cloud Threat Detection Lessons from a CISO

Security.

Hi there. Welcome to the Cloud Security Podcast by Google. Thanks for joining us today. Your hosts here are myself, Timothy Peacock, the Senior Product Manager for Threat Detection here at Google Cloud, and Anton Chuvakin, a Reformed Analyst and Senior Staff in Google Cloud's Office of the CISO. You can find and subscribe to this podcast. wherever you get your podcasts, as well as at our website, cloud.withgoogle.com/cloudsecurity/podcast. If you enjoy our content and want it delivered to you piping hot every Monday, please do hit that subscribe button. You can follow the show and argue with your hosts on Twitter as well, twitter.com/cloudsecpodcast. Anton, we have the CISO of one of the planet's largest financial and cryptocurrency companies today. We have a CISO who is the only person I've personally observed destroy another director's computing equipment as a way of getting their attention, and one of the most deep-down loving but surface crusty security people I've had the pleasure of working with. I am so excited to have Jim today.

Yes. And you have to wait for the story of the equipment destruction until the very last few seconds of the episode.

Yes, listeners, you have to hang in there.

Yeah, hang in there. So, this is actually fun because we wanted to maybe not surprise him, but like maybe a little bit scare him with challenges about how he does cloud security versus how he does on-prem security. But I felt like he really gave us a set of really solid answers for people, useful for people who are balancing the cloud stack with like regular stack, on-prem stack. I don't know what's the term.

Yeah, yeah. No, I mean, it's really interesting because he's not the CISO of cloud. He's just the straight CISO of the whole thing. And so it was really wonderful to have him talk about the challenges that he faces in trading off investment for Thinking about continuity of support, thinking about how issues move from one side to the other. So listeners, this is a really fun episode in my opinion, because we really get an honest, unvarnished, behind-the-curtain look at how one of the biggest companies on the planet is thinking about defending themselves. So with that, let's welcome today's guest. I am delighted more than you can imagine to introduce today's guest, Jim Higgins, CISO of Square. Or Block, depending on what we wanna call it. Jim, you were at Google for a very long time, and before you left, we actually sat near each other, and you sat in the org chart between internal security and GCP. Now that you're leading a major financial services company, how are you thinking about prioritizing your time between protecting your on-prem and corp world versus protecting your cloud resources?

Well, you know, at— it's Block, by the way.

Block.

And we have a subsidiary called Square now, and Cash App, and Tidal, and TBD, and a few other companies. And we acquired Afterpay. So, I'm responsible for the information security of all of that, which means it comes with, you know, still data centers and cloud, right? We're not one of the lucky ones that are 100% cloud for our production environments. And then, you have your corporate environment as well. When I look at the differences, we need to allow developers to be able to execute locally on their laptops. And that means it's very difficult to lock down a laptop, you know, or make it impenetrable. So we really do rely on zero trust. We do have— I think it's the only one I know of where Google BeyondCorp is deployed from the original deployment. So we do have Identity Aware Proxy and things of that nature, and it's super beneficial. Matter of fact, it's, you know, one of the reasons I took the job.

Hmm, so you wouldn't have to learn new things?

No, no, it's because I knew it was secure.

Oh yeah, that makes sense. You don't want to be the kind of CISO who loses.

I don't like to get popped, but you have to assume you're gonna get popped, and if you don't, you're gonna have a bad time because poppings happen all the time. It's just a matter of, you know, the severity of it.

Yes.

I don't want to speak for Google, but I did work at a company that was large, and I worked there for 10 years, and our instances on the edge would get popped a lot because they're unmaintained, like test instances.

Yes.

And they overnight become crypto mining instances. And guess what? It's the same out in the real world, real world being, you know, other companies. And, you know, it happens. It's just, can you contain it? And so we operate on a policy of containment when it comes to the endpoint. So our focus is really on You know, not so much like, do we have the right risk metric or anything of that nature? We are focused on what is the data, how does, you know, that we're protecting, what is that level of data? You know, is it super classified or is it public information? You know, so we have like 4 levels of data security. We look at how does that data move? Does it move from where it's written to into a SaaS provider or a data lake or something like that? Because that's equal in severity. What kind of controls do you have there? And what kind of monitoring do we have in place? And then on-prem is on-prem, which is just the same thing except for the tooling's a little different.

So that's pretty cool, Jim, that you're really using BCE, you're really using IaP, and you're using something that's delivered from the cloud as part of your on-prem posture and control. That's great.

Yeah, I think it's a sane approach, right? It's just like, It's one of those, why is common sense so uncommon? Like, it's what a sane person would do. It's just like very few actually do it.

I think that, yeah, no, you're right. And I was benefited by having a great team and joining. So I had all the benefits of Google, right? And then I had the benefit of the same mentality in the company that I went to. So you're absolutely right. It's like keeping people on the fundamentals. Do not get lost in risk. measuring and things of that nature, although it's important because you have to find a way to describe what it is you do for the company, right? In some kind of monetary sense, or metric or something. How does a company know you're doing your job and if you're getting worse or better? And it can't be compromises. Because if it's compromises, you don't want to be that person.

That's a very binary outcome.

Well, it's like, I've only had 5 compromises this year. And it's— that might be a—

It doesn't have the same, quite the same zing, right?

If you're reporting Is 5 good or bad?

Well, that depends, right? If 5 reportable compromises, yeah, they're gonna be talking to the CEO at that point. But the—

5 apocalyptic ones probably would be hard to do, right? If they're really apocalyptic, you can't do 5.

But yeah, so it's back to common sense. It's like, what are you protecting and is it protected and how do you know? It's simple.

I love that answer.

Yep. Me too. That's why I came up with it.

Yeah, that's good. So, let's talk about your cloud environment and threat detection. How are you thinking about threat detection in cloud? Because our listeners love talking about threat detection.

Oh, I'm sure they do.

So do I.

Again, it's more common sense.

By the way, actually, sorry for interrupting you, but this is like, sorry, my ex-Gartner brain kind of pops up and says, yes, common sense, but some people say using the same tools you used before is common sense, and others say using cloud-native tools isn't common sense. They're both common, but I'm not sure both are sensical.

That's fine. What I was getting at is, you know, as I look at a standard bare metal environment, or a corporate environment, or a non-cloud environment, what do I wanna know? What are my processes doing? What's running? Is there anything abnormal going on? How do I collect all that data, put it into a pipeline, and react to anomalies? In a timely manner. Because if someone's in your system and they've popped you or about to, it's minutes, maybe even less. I mean, there's a lot of automation around, you know, doing the easy compromises out on the edge of the cloud, which is what I was referring to earlier around crypto mining. So, it's the same for cloud, except for cloud, you're exactly right. The tooling is entirely different. The scale and the complexity is significantly larger. You have many microservices, you have containers, you have storage of all different kinds, and you have a lot of communication between all of that. You have developers who have the ability to start instances and stop them. And so, you have this very, like, living organism that you have to manage. you know, protect and manage. But the most important part goes back to the fundamentals. What processes are running? What is your network doing? What traffic is flowing? Does something look abnormal? Get that into a pipeline and send it to a SIEM where someone can respond or software can respond.

Mm-hmm.

You know, automation is super key. I'm a big fan of it. You know, automatically shutting down an instance that is not a P0 workload Like a supercritical workload, should not be a big deal. Like, we don't know what's going on here, so contain it real quick and let's have someone look at it.

But you're describing both similarities and differences between on-premise and cloud, it sounds like, because, say, collect data from sensors, whether it's slightly different sensors in the cloud versus on-premise, and then send it to a SIEM. At that level, it's kind of the same, right? I mean, you may use a different SIEM. I don't know if you do or not, but sometimes, people who approach cloud as a rented data center, and they follow the on-prem blueprint, kind of arrive at the same spot. They just make more mistakes along the way, right?

Yeah, highly probable. I would agree. There's a benefit, and there's a not-so-great benefit from, again, a common-sense standpoint. Putting everything into cloud makes monitoring and management of security tooling and observations A hell of a lot easier.

So that part is the simplicity.

My entire business is tied to your hypervisor and how secure it is, you know, or your security, you know, the underpinnings of that infrastructure, right? The cloud infrastructure. We're putting all our eggs in one basket. And so I'm betting the farm, quite literally, that, you know, a cloud provider has their stuff secure.

For us, that security is an existential risk. If we get that wrong, there's no do-overs for Google Cloud on that.

Well, there's no do-overs. And yeah, I agree. And I can go into great detail about, you know, what if you have critical infrastructure on there for government? What if you have many governments on there? What if you have many financial systems? It just gets bigger and bigger and bigger.

Yeah.

And it does come up as concentration risk. That was the comment I was trying to make, is that we are— somebody here at the office said this CISO is writing a paper specifically about concentration risk because some people show up at our door and say, this whole eggs in basket thing, I just can't get over it. Like, I can't stomach the whole eggs in one basket. I know you have the best basket on the planet. I know it's protected by the best of the best of the best of the best of the best security engineers in copious numbers, and they're all very motivated and skilled, but it's still one basket and I can't get over it. So, this is still something we need to address more formally, right? We may have the most secure basket on the planet, but it's still one basket.

Yes, and it's a concentrated place for all attackers to point their weaponry, so there's no diversification.

That's one thing I've always wondered about this GovCloud concept. Why would you build a separate dedicated GovCloud location for everybody to go attack?

But you have to use inferior tech for that, like some of the competitors do, right? Because they use outdated cloud tech in a little GovCloud, right?

Anyway, we're getting, uh, well off topic. I want to talk about the automation piece, Jim. You said just now, oh yeah, if there's something funny on an instance, we can just kill it as long as it's not critical. Do you really do that? And how do you keep the engineering team from, you know, coming to your office and finding you?

We design for that. We don't do it very often. And so, I'm speaking more rhetorically. You really have to know your services. You have to know, you have to have, well, I'm gonna use the word perfect mapping of your services because you can take out, you know, with so many services, Maybe one service breaks the chain and takes out prod, right? Then, you know, you did it because of a P1 vulnerability or some— what you thought was an abnormality, but it wasn't. And then you have a bad relationship with engineering. You don't want to be, you know, the job of a CISO and the job of the security team, and people will disagree with me about this, is not to secure things. I know that's controversial, but hear me out. It is to understand what the business needs, what the business priorities are, and what risk is generated by that. And then how do you find a way forward? You know, if you focus on, you know, vulnerability management is extraordinarily important and fundamental, but if that's all you focus on, you're going to slow the business down. You're going to miss a bunch of stuff.

Right.

And you're going to be seen as the organization that is more of a checklist-oriented Sometimes it's referred to as the no team, that can be overridden. You want to be a partner because you are a partner.

Yeah.

You are a partner. You're all in the same company. You may have to make controversial decisions, but it should be in line and not out of line, right? It should— you should be in the room with the engineers and leadership having a discussion on a common stance and not, you know, emails or tickets flying back and forth with thou shalt never. Because that's fundamental, which brings up my joke about policy. So, it's actually, you know, not a joke. And forgive me, compliance people, but I don't believe strongly in written policies as an effective way to manage a security program.

Because it results in paper-only security, or why?

Unless it has— there's no such thing as an effective policy unless it has an actual technical control behind it. So, don't write policies unless an engineer can write a control to it. Otherwise, it's—

That's another common-sense point that people have been arguing since the '90s and just not freaking doing it. Like, what's up with that? Sorry, I'm getting incensed about it because I've argued the same point that if you write a policy and then you don't do it, you're actually worse off because you can be found negligent if something blows up. They would point at you and say, you idiot, you wrote the policy that you're gonna do X.

Where is X?

And you're like, uh, no X, right?

Well, there's also, I mean, there's legal factors, right? You have to have a policy to have insurance. You have to have a policy to be SOX compliant. You have to have a policy or a set of policies for just about any audit, PCI, everything. And so, I'm not saying that there's a negative attribute. What I'm trying to push for, well, there is, but what I'm trying to push for is a control behind that policy. If you're serious about your policies, have a control behind it. And then when you go to write that control and implement it, you're going to learn some things. I promise you that.

I really like that. And I really like your perspective about what a CISO is for. It's very in line with the viewpoint that Alyssa Miller put forth when she was on the show.

And I think it's also a very real, very reality-based view.

You mean Allison Miller?

No, Alyssa Miller.

Alyssa Miller, the different Miller.

Yes. Oh, there's 2 of them. And they actually get confused on Twitter all the time.

Oh, yeah. Well, I sat next to Allison for years.

Yeah, at Google, right?

If you can hear me, Allison, I miss you. I hope you're doing well.

She's at Reddit now.

Yep.

So let's think about, as you've been growing your cloud footprint, what's been hard to get right from an org perspective in cloud? Is your org structure different for cloud security versus on-prem security?

Okay, so when you say org structure, what do you mean? Do you mean like projects and subprojects, or do you mean like, how do I—

Team structure.

The people.

Skills.

The people, the teams.

Yeah. So again, I'm coming from an environment which is, we're in the transition from bare metal into cloud, right? And so that means our organization is adapting. I don't have the benefit of building an organization that is immediately cloud-centric. So we have a specific cloud security team that works with our infrastructure team very closely. As a matter of fact, the entire division is called infrastructure security, and it's run by A wonderful gentleman named Paul Friedman. And, you know, he has BeyondCorp and network security and cloud security and, you know, software supply chain security and a few other things. And actually, he doesn't have software supply chain security, but I'll give him the benefit since I'm live right now. And, you know, our cloud entities, we have some of the best engineers I've met. Including a gentleman by the name of Scott Piper, who's fantastic. And they spend a lot of their time on the infrastructure level building what I would refer to as more horizontal than vertical, right? So more, where can we have the most bang for the most buck? And what I mean by that is I have 7 different business units I have to manage. There's 7 different companies. And some are on different cloud environments, some are on the same. It's just like, you know, alphabet.

Mm-hmm.

And, you know, pointing your resources and your team and your organization towards where you're going to have the most lateral effect across all 7 or 8 is more beneficial than going vertical in, you know, like, we're going to make this one product really, really secure. However, if you do go make one product really, really secure from the bottom up, the other products can go steal that. And then it benefits that way. So I just spent 3 minutes contradicting myself.

Well, honestly, for you, that's not a bad number of minutes contradicting yourself. Yeah, not bad.

But I want to bring up, bring this, bring some numbers behind it and kind of talk metrics perhaps a little bit, because to me, metrics matter for like a grand scheme of things. How do you measure security? And of course, I'm not going to ask that question because I feel like it's going to be another 6-hour podcast. But how do you measure security in the cloud? Like, how do you measure whether automation is working? Are you saving time, money, you know, annoyance? I don't know. What's the approach for measuring stuff?

My metrics are so good that they're classified, and I'm not sharing them with the world.

You can always uplevel 3 levels and say, I guess if you uplevel the 3 levels, you'd say, I measure the important things.

Oh, I promise there's a lot of people laughing right now because they know I'm full of crap. So, obviously, compromises in cloud, You can look at a number of instances versus number or number of vulnerabilities.

Mm-hmm.

You also really care about more— it's weighted, right? You care more about what's exposed to the internet versus not, and the sensitivity of the data that it manages. So, we find a way to look at, you know, I know I just kind of made fun of vulnerability management, but it is one of the key ways to show, hey, this business unit has 14 P0s and this other one has 0, you know, we kind of know where to go point our resources at, right? From that standpoint.

Hey, on that standpoint, we've got a really cool capability coming in about 2 months here where we're going to be able to tie issues we're seeing in cloud back to the importance of the assets that those issues affect and the degree to which those issues open or don't open a pathway between the internet and that resource.

Yeah. Can you do that inversely for insiders as well?

Uh, we can talk about that.

If you can do that, that would be cool.

So that— No, we should really talk about that.

That's not about path to internet, it's about how many people have access to this. And, and that's very complex, right? And there's, you know, I, I get a phone call every day about a startup that's trying to solve that, and it's not easy because A lot of them just do cloud-native, and I know this is a cloud podcast, but my risk is more than cloud, right?

Yeah.

And attackers don't see it. Well, I'm just going to stay in the cloud because this is only a cloud hack. Nope. It's going to be laptop to God knows what to what. Yeah.

This is what I learned from talking to the Mandiant guys. They almost wanted to steer me away from cloud security because if you're hacked on-premise and then use the creds to go to cloud, Can you really say it's a cloud security issue? If they hack the freaking, I don't know, Active Directory server, I don't know, random example, and then use the creds to log in into your cloud environment, then is this a cloud?

It's just another fancy word for a different type of infrastructure that's all the same as it was before, right? It's just somewhere else.

We typically frame it as—

Jim, before we go down that rabbit hole, I'm afraid I have to ask our traditional closing questions.

Yes, sir.

Do you have recommended reading for our group? And some people will be surprised to learn that you do know how to read.

No, I don't, actually. I have my dog read to me. Charlie, come here. What books am I reading right now? What books?

I don't know.

I don't know why this pops up to my head first, but it's Mitnick's book.

Oh, yeah, sure.

His autobiography. Especially with the recent Uber compromise.

Yeah.

Social engineering is still the most effective way to pop an environment. The human is always the strongest and weakest link. And strongest means if you, if you make them educated and informed and part of the security extended family, they're going to be of great service. And if you don't do that or treat them poorly or something to that effect, and I'm not saying anybody would do that, But if you did, they're going to be less effective, and then you're going to get popped.

That's a great recommendation for reading. Do you have one weird tip for helping people improve their cloud security outcomes?

And don't say use common sense, because I think you already used up that one.

One weird tip for improving cloud security outcomes. To manage cloud security and cloud security outcomes, you have to be prepared for scale and growth like you've never seen before, right? And you have to become very comfortable with small compromises. And, you know, it's weird for me to say, but it's going to happen. So it's how you plan for that. Prepare to get popped.

Mm-hmm.

Now, your press people might be sitting there saying, well, he just sat there and described if you move to cloud, you're going to get popped. But that's not what I'm saying. What I'm saying is with opportunity comes some, you know, increased risk and automation and a robust detection pipeline will be the most beneficial thing that you can do. That's not weird. Always wear a green shirt.

I don't think that sounds weird at all.

That's how you'll survive. Always wear Hawaiian or green shirt.

It's very important. Jim, thank you so much for joining us today.

I don't get to tell the story about wireless mouse. So, Jess LeRoy sat near me, and I often was hunting him down since he was the head of all the product management for cloud security at the time. And I got frustrated, couldn't find him. And so, I went to his desk, and I took a pair of scissors, and I took his wired mouse and turned it into a wireless mouse by cutting the cord. And I left him a note saying, I've turned your mouse into a wireless mouse. Please come find me. And it worked. He came and found me.

Listeners, I saw it happen.

Yeah, you were there. Yeah.

I was sitting there. It was amazing. I'd never seen such a thing. Jim, thank you so much for joining us today.

It was a pleasure, and it was great seeing you all.

And now we are at time. Thank you very much for listening and, of course, for subscribing. You can find this podcast at Google Podcasts, Apple Podcasts, Spotify, or wherever else you get your podcasts. Also, you can find us at our website, cloud.withgoogle.com/cloudsecurity, Please subscribe so that you don't miss episodes. You can follow us on Twitter, twitter.com/cloudsecpodcast. Your hosts are also on Twitter, @Anton_Chuvakin and @_TimPico. Tweet at us, email us, argue with us, and if you like or hate what we hear, we can invite you to the next episode. See you on the next Cloud Security Podcast episode.

Bye. Security.

Security.