EP136 Next 2023 Special: Building AI-powered Security Tools - How We Do It?

Security.

Hi there. Welcome to the Cloud Security Podcast by Google. Thanks for joining us today. Your hosts here, as ever, are myself, Timothy Peacock, the Senior Product Manager for Threat Detection here at Google Cloud, and Anton Chuvakin, a reformed analyst and senior staff in Google Cloud's Office of the CISO. You can find and subscribe to this podcast wherever you get your podcasts, as well as at our website, cloud.google.com/security. If you enjoy our content and want it delivered to you piping hot every Monday, please do hit that subscribe button. You can follow the show, argue with your hosts and all the other listeners of the Cloud Security Podcast on our LinkedIn page. Anton, this is a special just in time for our next conference this week. And like many technology conferences this year, AI is a core theme. And today's guest, I'm really stoked about because he's been at the forefront of leading the engineering charge on building not just security for AI, but AI for security.

Indeed, this is a very fun one. And I feel like today some of the listeners may expect the deep secrets about how we really did it. And actually, yes, some of it will be revealed, but at the same time, it's a really good conversation about somebody who is in the middle of building AI for security and of course securing AI as well, who is a senior leader in the organization.

I thought what was really interesting on that deep secrets front was when he broke down the problem, it seemed so simple.

Yes, at the risk of spoiling, I would not say a word. I think that's exactly right, yes.

And I think maybe that's a mark of good leadership and good communication that you can take this hairy, intractable thing, securing AI, AI for security, and make it sound like it's not rocket surgery at all. And so maybe with that, listeners, let's turn things over to today's guest. Today we are joined by first, an expense approver in chief, and second, one of the best VPs of engineering I've had the pleasure of working with here in Google Cloud Security. Eric Derr, thank you so much for joining us today. This is a fun episode because we get to talk about one of our favorite topics of skeptical eyebrow-raised contention, AI. And now I understand this week at Next, you're talking about AI. So tell me, with my eyebrows cocked and ready to go, what are you talking about?

So I'm gonna talk a little bit about what we're doing with security use cases and AI. And one of the exciting things we announced at RSA is that not only are we taking Google's generative AI and building that deeply into the different security products, but on top of that, we realized there's kind of a gap in the market. There was nobody really looking at the security use cases and building an AI platform for everybody out there who's trying to protect themselves, their infrastructure, build security applications, et cetera. So we're well on the way to that, and I'll give an update on everything we're learning and have some partners and customers come on stage with me and talk about it.

Ah, that's really exciting. So a couple of parts of that then: how we're using AI, how we're building a platform for AI for security. Was there a third piece? 2 pieces? 3 pieces?

I think there's 2 pieces.

2 pieces.

We're using it everywhere and we're building a platform.

Okay.

That works well, but I think that there's also a subtle silent third, right? You're also dealing with securing AI, right? So it's almost like when somebody wakes me up at 3:00 AM and says, Anton, AI and security, I think first about using AI for security, but I also think of securing AI. How do you organize it? Like, ultimately, I don't have to engineer anything, thank God.

Thank goodness.

But you are a VP of engineering, so—

Listeners, you're very lucky Anton doesn't engineer anything.

I think there's a couple of things. First, you got to ask the question, Are you building your own AI system or are you building on top of, say, Google's AI system? And if you're building your own, wow, you got a much longer list of things you gotta worry about. And obviously we're incented to people to build on ours, but there's a good for the planet kind of thing of saying, look, it's actually hard to do this well. And building on top of a trusted platform like we have with Vertex AI eliminates 90% of what you gotta worry about. So that's the first thing I'd say. The second thing is, if you zoom in and then say, okay, what are the classes of attacks against AI? It turns out the vast majority of them are conceptually at least identical to securing any other piece of infrastructure. And so, for example, some people are talking about the concept of hypnotizing LLMs, which is a malicious insider implanting a bunch of information so that the LLM will do something malicious. It is maybe a little bit out there, but if you just say, look, if you have a malicious insider, what are the things they could do? Well, there's a long, long list of things they could do if they're in the right place or the wrong place in a company, right? And so, it's a little funny to me that people are getting spun up about that particular use case when you have such a long list of things a malicious insider could do.

There's also decades of experience dealing with insiders.

That's correct.

Is it calling for new solutions or is it just we're realizing that our old problems exist here too?

It's more the latter, I think.

Really?

And in some cases it might be, hey, we need to extend a control to a new surface area, right?

Sure. That's normal.

That's work, but that's not rocket science.

Yeah.

That's just the normal course of doing business.

Just normal security problems, not rocket security problems.

Right.

Here's another example. People are worried about LLMs generating code and what if they generate insecure code? Well, it turns out humans generate insecure code all the time, right?

Yeah.

If the LLM for some reason was generating code that was 10 times as insecure, that might be a real concern, but we're not there. Like, that's not the reality we're seeing. And it's also like, we're still in the evaluation phase. We're not just letting AI loose to generate code that gets automatically shipped to production, right? Like, that's—

We're not?

No, I don't think so.

We are not, but some others are apparently.

Some others are, right? So I'm not trying to say there aren't unique issue. Like, I think prompt injection is a real kind of new use case. But even there, I'd say, well, look, we as an industry spent years on SQL injection and figured out a set of techniques that, you know, it's not that that's never a problem, but it's certainly not the problem it was 20 years ago.

No, definitely not.

And I think that's similar enough that I think it won't take us 20 years to figure out how to really sand off the corners of prompt injection. We've got a little ways to go. We've got some work to do there still. I think a lot of it is fear of the unknown because for a lot of us, it's early days. I don't mean this with the slightest bit of judgment at all. There's just a lot of learning that we as an industry have to do to wrap our heads around the use cases as well as the proper ways to use these things and make them safe.

I think one thing you said in there really struck me about the AI writing code that's 10x as insecure as human beings. I don't think it is. We have almost similar debate, perhaps appropriately, happening in my hometown right now around self-driving vehicles. And I think it's pretty clear that the self-driving vehicles are not 10x more dangerous than human drivers. Are they going to make mistakes sometimes? Yes. Do human beings get drunk and text and all that? Absolutely, yes. And so I wonder, like, how do we as a society start to hold these things to the correct standard rather than an arbitrarily high standard? And maybe I don't know if that's a VP of engineering question or not.

Well, you know what it reminds me of? In a past life, I was a defender and led the team that ran the SOC for a big company that rhymes with Flasjer. And, you know, one of the things that, like any big cloud service provider, you deal with a lot of fraud, right? People attempting to misuse the resources. And there were definitely a few situations there where And this was fascinating to me. I think it's a human behavior thing where we came up with a set of automation that was actually as effective, if not more effective, than the kinds of error rates that a human has. But that actually wasn't an acceptable bar for the business.

Oh.

It was, for some reason, more palatable to have a human look at something and double-check It might have been automation-assisted, but there was some kind of comfort that the people took from, okay, a human's going to look, and before we disable that subscription, make darn sure that it's not a false positive. I just think that's fascinating. It was fascinating for me at the time because I could understand the perspective and even say, it's not actually logical, but man, there is something nice about having a pair of human eyes. I think there's a whole lot of similar kinds of questions ahead of us on the GenAI frontier, places where I think we'll get, for many, many use cases, AI to be as good or better than humans, but especially where it's hard to measure the human error, and even where it's easy to measure, but there might just be a kind of a cultural resistance to turning the keys over to the car. Or the AI.

Is it the case that the full automatic mode has been proven to be better than humans, but there is still hesitation to go full auto? Like, roughly, that's the scenario you're describing, right? When you already have solid data that full auto is better.

Yeah. I haven't seen this yet in gen AI, but I'm saying in a different world, in a different life, I saw something like that play out with more traditional automation. And so I think we'll see some of that in the future. And then in a way, like every organization has to make their own decision about where their risk tolerance is, right? And what kinds of errors are they willing to risk? And that decision is best done by the people who understand the business and are accountable for the business outcomes.

So I guess on that note, how do we help business people think about trusting AI? I mean, how do we work them through that? What's the words we should use? What are the concepts we should use? What does that conversation look like at the leadership level? How's that going to go?

I think that primarily it's a conversation and it's education. And in some cases, it's not helpful to try to educate people because it's easy to come across as paternalistic or talking down to people, which is not at all helpful usually. I certainly think just anecdotally in the conversations I've had with customers over the last 3, 4 months, it is top of mind. And there's a lot of angst. And one of the things I think, of course, I'm a fan of the approach we're taking in Google Cloud around this. And one of the really nice things is that Vertex AI is not, they're not starting 3 months ago on the enterprise security problem, right? They're building on top of many, many years of good practices in Google Cloud. And in some cases extending those, I mean, just examples of data residency controls that are built into the platform and things like that, that In most of the conversations I've had, people are like, oh, I'm worried. And then in some cases, they are only familiar with OpenAI.

Hmm.

And not a slam to OpenAI. I have a ton of respect for those folks. And I think they'll add more enterprise security controls over time, but they're starting with a platform that isn't fine-tuned for enterprise and doesn't have the years of experience of iterating on that. And so I think it's helpful for us to be able to say, hey, look, here's a set of things you might be worried about. You might be worried about what happens with your data, and are you in control of your data? And here are the controls we have, and here's how we've extended them to AI. And so for a lot of customers, that's very reassuring.

Mm.

It's kind of connecting the dots because I think people are just— they're still trying to wrap their head around lots of pieces of this.

Mm-hmm.

And so just like, hey, here's some pointers and some stuff to take a look at, and please give us feedback. Because it is still early days for all of us in GenAI. And that helps. But I think that it kind of goes back to what are you willing to put up with in terms of risk. It's also what pace do you want to take, right? And some people are ready to go 60 miles an hour. And some people are ready to go 10 miles an hour. And again, that's OK. People need to go at the pace they want to go.

That's fair. I've certainly seen some vehicles on the streets here go at the wrong pace. So I think that's good encouragement for people to keep in mind where they're safe at right now. On the GenAI early days front and securing it and talking to people about, say, data residency, have you seen a good model yet for helping people think through their threat models and make that really concrete and really break down the problem end to end for them?

Google published a SAFE framework, which is worth taking a look at. And if you look at it, kind of to my earlier statement, it takes the best of good practices of many years of securing various systems and then kind of says, how do you apply that same thinking and extend it a little bit? when it comes to AI. And so I think that's a good place to start. I think for a lot of the customers that I've talked to, they're still figuring out where they want to use GenAI, which is kind of question one before you start worrying about, well, what's my role in it? What tech am I building? What data am I using? And therefore, what's my threat model that applies to that?

I think the usage comes up a lot in these discussions. That's a really good point, is that you can come up and say, Hey, let's have a drink and talk threat models. But instead you say, hold on, how are you using the system? What are use cases? What is the business? Are you in regulated financial industry stuff or are you in making pictures for people to have fun with? I can see how threat model absolutely hinges on this. And so it's almost pointless to discuss.

Wait, Anton, are you saying we're allowed to drink while talking about threat models?

It's a must, actually, in some circles, but it depends on the threats, I guess. Which brings up another example about evolving to trust. So you brought up a point about use cases using AI for security, and you kind of like mentioned this whole generating code, running it. So people ask about what are the best use cases, but in your mind, what are some of the worst use cases you can imagine? Like, I have seen a vendor use GenAI to produce, I think, Terraform code— not Terraform, one of the scripting languages code that you can then run on prod systems to block threats. That freaked me out. Not because I don't believe it's possible in the future, but I think it's absolutely not a good idea today. So any other, hey, let's not do it with GenAI kind of use cases that you can think of?

I have two minds on the subject. On the one hand, I would say my goal is to get people to thoughtfully use GenAI because I think it is a useful tool. And almost any tool can be misused if you use it thoughtlessly.

Yes.

So that's one thought process. I think the key to me is you gotta— this is gonna sound really basic, but you gotta think about what problem you're trying to solve, figure out how you're gonna approach it, try it out, and make sure that the tool is actually useful for the problem you're trying to solve. And if you skip any of these steps, then you have a problem. And so I'll give a really good example of this.

Mm-hmm.

One of the use cases I think is exciting is GenAI inside of automated detection pipelines to help with efficacy of various sorts, right? We have lots of techniques we use inside of automated detection pipelines, just as an example, but like GenAI is a new one. But just like the other techniques, you didn't just think of it, code it on a weekend, and run it with no testing because you wouldn't like the outcomes of that. Similarly, You wouldn't want to do that with GenAI. And so in some cases, the testing is a little harder because GenAI is harder to predict all of the things it might do.

That's a very nice way to say non-deterministic. I like that.

Let's use a more positive word. Let's say it's creative.

Yeah.

Not to give it human characteristics.

Yeah, that's right. Do we have some kind of like a don't anthropomorphize AI We'll get a new swear jar for that.

We'll put it right next to Anton's blog, Anthropomorphizing AI.

Yeah, but I think that it's just more fun to occasionally anthropomorphize AI. Sorry. So at the risk of anthropomorphizing AI one more time, in our marketing materials, we do say for security, we don't just use general models for LLM. We have a security-trained, or we teach AI security. That's anthropomorphizing. What does it actually mean? Like, you were involved with a lot of stuff, like how are we teaching AI security? And what are some of the challenges there?

I think this one's my fault. I think I was the one that first said, we're teaching the language of security to the LLM. I've talked about this a couple of times, but I'll kind of repeat quickly. The journey we went on in the security teams when we said, hey, there's some really cool applications of GenAI. And just like everybody, you start You know, you know some things when you start, but then you learn as you go. And what we found is that the foundational models, while amazing, like our foundational models that are coming out of Google and DeepMind are amazing, but they were not always amazing for specific security use cases. And the categories in which they were not amazing is in places where there's significant proprietary information that's not available in the general training sets. A great example of this is YARA-L, which is the language we use in Chronicle for queries and detection authoring. There's not a ton of resources on the web, and so it's not normally fed into the training data of the LLMs. So, of course, the LLMs weren't amazing out of the box with YARA, but we said, boy, that's a use case we think is super important, that if we could take natural language and then output really good YARA-L, man, that would be a productivity boost and a training boost to all these analysts who are out there trying to get their jobs done, right?

Mm-hmm.

But the good news is we have a whole bunch of samples of YARA-L. So it's just a process of feeding that information into the LLM to better train it. And now, you know, now it speaks that— this is where I like the language metaphor. Now it speaks the dialect of YARA-L, but that doesn't mean it speaks the dialect of SPL, which is Splunk's query language. And this is the ongoing process as we're working with more and more partners who are betting on our platform, is in some cases they'll come to us and say, here's a use case, and we say, hey, this works great. And in other cases we say, hmm, okay, we need to bring some more training data in to fine-tune the ability of the LLM to do this use case.

That's really interesting, and I think we mostly stayed away from making the LLM Too relatable or too cuddly sounding.

Cuddly.

I hate to do it, Eric, but I have to ask you our traditional closing questions because I think the 3 of us might be able to spend all day yakking about AI. First, do you have recommended reading? And then second, aside from attending your talk this week at Next, do you have a tip for audience members who want to get even more up to speed on security and AI?

I think that the recommended reading— I have nothing work appropriate, but I have a fun book that I recommend, please, which is called Hail Mary. It's about a threat to the solar system and an expedition to another star to try to stop the threat. It's actually very good sci-fi reading, but good kind of escapism. Think about other things than GenAI.

Would you call it good sci-fi, or would you call it very enjoyable sci-fi? Because I really enjoyed that book, but I'm not sure it has, like, the depth or the emotional staying power for me that, like, Neuromancer does.

It's all relative. And so, sure, on the continuum, I mostly read trash sci-fi.

Okay.

And so relative to trash sci-fi, Hail Mary is good sci-fi. I agree with you, Neuromancer is a few click stops further on the continuum.

That is a high bar.

That's—

I set a very high bar there.

It's a high bar. Yeah. In terms of recommended reading, I mean, man, there's just a lot of really good content out there. I definitely would say there's a number of good blogs. that Google Cloud is putting out. I think there's a lot of interesting stuff happening in open source that it's important to kind of pay attention to what's happening in the open source community. But I can share some more offline, some specifics, but that's what I'd recommend.

Awesome. Well, Eric, again, thank you so much for joining us today. I'm looking forward to hanging out with you next week at Next and talking about all the trashy sci-fi we enjoy together.

Sounds good. Thanks, guys.

Perfect. Thank you. And now we are at time. Thank you very much for listening, and of course for subscribing. You can find this podcast at Google Podcasts, Apple Podcasts, Spotify, or wherever else you get your podcasts. Also, you can find us at our website cloud.withgoogle.com/cloudsecurity/podcast. Please subscribe so that you don't miss episodes. You can follow us on Twitter, twitter.com/cloudsecpodcast. Your hosts are also on Twitter at Anton_Chuvaikin and TimPico. Tweet at us, email us, argue with us, and if you like or hate what we hear, we can invite you to the next episode. See you on the next Cloud Security Podcast episode.

Thank you, Roni.