EP238 Google Lessons for Using AI Agents for Securing Our Enterprise

Security.

Hi there. Welcome to the Cloud Security Podcast by Google. Thanks for joining us today. Your hosts here are myself, Timothy Peacock, the Senior PM for Detection, UEBA, Risk in Google SecOps, and of course, Anton Chuvakin, a reformed analyst and senior staff in Google Cloud's Office of the CISO. You can find and subscribe to this podcast wherever you get your podcasts, as well as on our website, cloud.withgoogle.com/cloudsecurity/podcast. If you enjoy our content and want it delivered to you piping hot every Monday, please do hit that subscribe button. You can follow the show, argue with us and the rest of our Cloud Security Podcast listeners all on the Google Cloud Security Community page. Anton, show without you today, very sad, and especially sad because I think you were really looking forward to this one.

I kind of was, but also I almost expected that you would find a secret temporary co-host to replace me for that one time, like I did with some of the recent episodes. And so, I guess I'm not sure what I'm disappointed about, the topic or about you not having a— what's the term— fling with a secret temporary co-host.

No, don't worry. There was no secret replacement of you. You are irreplaceable, unlike me. So, today's episode, though, is really fun because we've got one of my favorite PM colleagues who's ordinarily based in Zurich, but happened to be stateside. And so we were actually recording this one in person together in the studio in Spear Street. So that was fun. And of course, the topic is great.

I mean, the topic being agents for security to me is incredibly exciting. And it's also exciting for multiple reasons, right?

It's not just agents for security.

It's not.

It's Google's use of agents for security.

Right.

That increases the excitement by roughly a factor of 100.

I think so. Maybe without any further increases in our listeners' excitement factor, Let's turn things over to today's guest. Today I'm joined in person in the studio in 345 Spear Street by Dominic Svirad, a senior PM for DNR AI and SecGemini. Dominic, we've wanted to do an episode together for years.

Forever. I think it's like 5 years in the making.

Yeah, this episode is a long time coming, and tragically, we're without Anton today.

Very sad today. Big fan of his writing, and so I'd like to see him, but next time.

Anton is on a boat, listeners, so you get another special Tim-only episode. So you've been working on all kinds of DNR at Google for a long time, but now you're on something really interesting to me, and I want to talk about AI agents for securing Google. It's pretty cool. So when you first started introducing AI agents to the SST folks, the seasoned security professionals, how did you— because I've worked with these people.

Yeah.

How did you start to build trust and overcome their deeply skeptical mindset? These are not easy people to work with, in my experience. So could you walk us through the first conversations and how you started to build that trust?

No, definitely. I think part of the issue that I found in discussing AI agents wasn't the fact that it's agents or autonomous. We like to automate things. We have been automating our security workflows for a long time. I think the difficulty that we found was the jump between machines that were previously excellent doers of human-defined plans that security engineers defined to now machines that would also be excellent planners in defining those plans and then execute on that.

Cool.

And when it comes to agents, I think that jump toward the planning, right, to giving control to a machine and say, you're actually going to develop a plan, that's the kind of the crux of the problem. Overcoming this, I'd like to say it was kind of my idea, but it wasn't. It actually came from the security engineers and some of our privacy folks. was going down to that level and saying, well, how do we not necessarily build trust on AI agents? Yes, of course, we want that. But how do we build trust around that fundamental technology that is going to be powering this, the LLMs and the generative AI capabilities?

Mm-hmm.

And so what we actually started doing, one of the recommendations that had come from our security folks was like, listen, Dominic, if you want people to trust the technology, they have to understand the technology. They've been working with automated systems for a long time.

Mm-hmm.

GenAI is new. The way to trust it is how can we actually integrate this deeply into our tooling, make it very easy for them to start trying different things out beyond just AI agents? Simple things— ask questions, interact with their data. Because if they can get a sense for how that works and how that functions, now they're going to be able to trust the bigger systems that you're building. And so we did that into our tooling. We actually integrated a generative AI chat interface. They can chat with their data, ask different questions. We had some things that we supported out of the box, other things that They can ask just generally. And what I saw was counter to what I had actually expected. There was a massive surge in utilization. I said, great, new feature. Of course, people are going to use it. That stayed consistently month on month and month and month.

Really?

Yeah. When I talked to security engineers, they said, before, yes, we had other tools, right? Of course, I could go to Gemini. Of course, I could go to NotebookLM. But I had to jump. I had to copy things over. Can I use it? Can I not use it? But it's there for me. I get stuck on something. I want to ask a question. Let me give it a try. Let me see it. Let me understand how it works. And that started the fundamental shift, actually, that started coming in people saying, hey, I kind of get this. Can I do more? What can this work—

Huh.

AI agent thing that you're building out, how is it going to help? And so I think that integration, like really helping people understand the core of the technology that you're going to build everything else on, that's fundamental. Like that— it's not going to solve all the worries and concerns, but it's— if you don't understand the technology, you're not going to appreciate everything else that's being built on top of that.

That makes a ton of sense. So when you started this out, there were surely a million problems you could have tackled first. How did you pick the right initial use cases for agentic AI in securing Google?

Yeah.

What made a good use case for this?

There are many of them. I think that's the challenge.

You look at it and say— Right. There's so much space here.

I can apply AI to everything. Part of the guidance I've given the team as we do this is, let's not just apply AI for the sake of applying AI.

No, we don't want to sprinkle AI pixie dust on everything?

I mean, we can, but we don't have enough time and resources to build everything that we want.

How do you pick the right use cases?

Yeah, so what we looked at is, well, our goal is here to deliver more security, right? We want to advance the state of security. What are actually the core bottlenecks that are getting in our way? And that's not as easy as it sounds to pick, because when you look at something like detection response, you may say, well, of course I'd like to have more coverage. Let's use AI to build out as much coverage as possible and accelerate our efforts there. Definitely a good application of something we want to do. But you ask the question, if I wave the magic AI wand and I got you 100% coverage, what would happen?

Yeah.

And you say, well, could our ops teams handle everything, right? Would we be able to maintain all those detections? And so that started kind of like, as you start pulling on that cord, you say, what's the true bottleneck? Where can we actually start, you know, really delivering value? Oh, let's actually start looking at what happens at ops. Let's start seeing how can we actually scale those teams to be able to handle more things that are going to be coming through.

Sure, sure, sure.

And so that was actually the starting point for us to look at. The other part, and this is a little bit maybe more basic, What do you actually have access to in terms of data? There's many things that you look at and be like, wouldn't it be amazing if AI could solve x? And you're like, do I have enough examples to showcase this? And I think the key thing there is AI is magical. It's not magic. If you don't have great examples of what it's supposed to do, maybe not the right starting point. Let's go back. Let's make sure we can curate the right amount of data and then revisit this afterwards.

I love the concept of focusing on the bottlenecks. Were there areas in your use case exploration that turned out to be no-go areas that you couldn't address, not for lack of data, but for maybe other reasons?

I think less of a no-go. It's more about where are you comfortable with deploying AI agents for. If you're developing something that's right in the middle of a firefight that is going to assist you with making very, very important decisions, are you at a position yet to have built that confidence to say, yes, we're going to deploy that in that area? And I think looking at this and making the judgment call and be like, all right, this is— First things that we're starting to roll out, where can it help? Where can it not help? Where is it maybe a little bit riskier? And kind of dealing with that and saying, we're going to address this later. Let's actually validate that we can do this in this scenario and then go and tackle the bigger problems later.

That makes a lot of sense. So as you're doing this, surely you're measuring the success and measuring your gains. So what metrics, what feedback mechanisms, how do you quantify the ROI here? Because I'm sure you're burning just like a ton of TPU cycles.

We look at 2 things. Of course, efficiency is one of those. I think that's a key metric, but I don't necessarily think efficiency is—

But what is efficiency here? Is it how many cases per day per analyst?

That's actually an open question, which is why we kind of measure things and how can we make you faster in certain situations, but it's not necessarily the core metric that we want to look at because it differs from people to people. The 2 parts that I look at is one, In terms of, are we advancing risk reduction, right?

Okay.

I mean, if you look at everything that we're supposed to do, we are advancing the security posture of Alphabet and our customers, right? We want to develop capabilities that'll push that forward. Are these new capabilities helping us close up gaps that we weren't able to close up before? Mm-hmm. Able to discover new things that we weren't able to discover before? That's kind of one area that we start looking at. The other area that I look at is, and it's kind of, kind of a thematic thing, once someone somewhere has discovered something, no one should have to ever do it again. Right. If we're going to really—

No patient twos.

No patient twos, right? And if we're really going to scale security engineering, you really want them focusing on the novel, the next things. And so when you start looking actually at the performance of your systems, you're like, is this actually helping advance that cause? Is it something that you've seen it once, you've seen it twice, fine. Third time, we can handle that. That's a nice way to start measuring how well are you actually advancing things, because you're now scaling their ability to cover more of alphabet.

As our former leader said, fool me once, shame on you, but fool me can't be fooled again.

Exactly.

So that's what we're going for with this is avoid fool me getting fooled again. That's really interesting. How, beyond the, you know, just conversational, magical hallway conversations about trust in it, how are you measuring the trust that the response teams are putting in these tools?

You know what? In a very, maybe overly simple way, ideas that come in.

Really?

Actually tying back to what I was mentioning before.

More ideas, more good?

More ideas, I think more trust, right? And so originally, you know, when I was mentioning we had launch this capability to let people start asking questions and exploring data directly, the amount of ideas that were coming were a couple, but not so much. As they started exploring more, they're like, oh, I like this use case. This sounds interesting. I've seen this work very well. And suddenly, you start getting this flood of concepts and ideas coming in where you see people start trusting the platforms and trusting the systems and trusting the technology more and saying, hey, this works. Can you do X for me? And I think without that trust, the ideas don't come. With trust, you suddenly start thinking, wow, this works for X. Can you do A, B, C, and D for me?

Uh-huh.

And that actually helps us understand So what were some of the initial successes?

I mean, we started very simple, right? When you look at just even generative AI as a concept, we saw that it was very good at distillation, right? Taking a bunch of information—

Summarization, yeah.

Not necessarily just summarization, actually taking a lot of data and finding some gleam of insight in terms of it, right? And so that could be a summary, but I think summary is kind of a broader term, and then translation, right? to code, code to text, text to code. Uh-huh. And so what we started doing is helping security engineers identify those points, right? Launching capabilities that allow them to look at an entire ticket, right? And say what has happened, the who, what, when, where, why, how, so they can make another decision. Look at code, understand is this code potentially malicious? Hmm. Look at files, right? And understand are they sensitive? Really helping bridge these kinds of gaps very often.

Sure, sure, sure. I could certainly see how you could accelerate a lot of analyst journeys just by looking at a file and saying whether it's sensitive or not.

Yes, definitely, definitely.

And you probably get all the upside of not having human beings look at sensitive files.

That helps quite a lot, right? I mean, a lot of the things that we're also looking for is like, how do we preserve privacy, right?

Yeah, sure.

Not necessarily just can we scale to do more security? 100%. That's something we want to do. But can we do it in a way that is privacy-preserving, enables us to look at things that we're supposed to look at and not necessarily have to get involved in everything?

Huh.

That's really, really interesting. As you're scaling these out, as you're scaling out these initial use cases, what interesting scaling challenges did you run into and how did you overcome those challenges?

I think part of this comes down to the fundamentals of testing, right? And so when we started this, a lot of it is you do an experiment, you gather certain bits of data and you say, great, very controlled, and you can start testing that. The reality, of course, is security is you can get the best representative dataset that you have, but security is varied, right?

Yeah.

Different things change in different times. And so how do you deal with that, right? The second you start doing something in pilot or you're doing a dogfood or you're moving toward launch, you now need to have a system in place or a process in place to be like, how can I actually gather the differences, analyze the differences, figure out what worked, what doesn't, and then feed that back into the agent and handle that? And I think that is a massive challenge when it comes to scale because That update process, I think when you look at the general automation systems, you'd go, you'd update the rule, still very complicated, but now you're dealing with a very fundamentally different technology.

Yeah.

That you have to identify the patterns and then like bring it back into it quickly enough so that security engineers are like, all right, great, this continues to work in that direction.

How do you update it? What does that process look like? Because surely it's not as simple as just updating your playbook to, you know, make a new API call.

No, a lot of it starts with data and data quality, right? And so your beginning point is to look at all the tickets that are processed to try and understand and get feedback from the users, what worked, what didn't, and then update your datasets, right? And then rerun it against Gemini and say, OK, how did it perform?

Hmm.

There isn't a magical wand toward this. I think this is something within industry that's still being worked out is, like, what's the most streamlined way to do this? And a lot of it is just get into the data, look at the data, analyze the data, and off you go. We've been doing some interesting things, actually. Some of the security engineers have been helping us build out where they actually use Gemini to help us analyze. this, right? And say, well, here's a bunch of, you know, there's a lot of tickets that had to be handled, right? We don't necessarily have the time to go and read every single ticket and understand it. Can Gemini go and look at it and say, oh, these are the differences actually that happened?

So you're at the point now where you're using Gemini to improve Gemini.

We're using Gemini to help improve our agentic workflows.

So the machines are helping themselves get smarter.

That is the direction that we're going toward, right? I think you—

That's a little scary, isn't it?

Actually, so the question there that you have is when we started looking at quality and like quality measurement, right? And so if you have AI start handling, let's say, tickets and managing them, then at one point other AIs pick up from those tickets. And what happens if you have an error that then propagates all the way through?

Sure.

And so the answer there is say, okay, great, let's build out an agent that does quality assessment, but how do you ensure that that agent also Right. Is running into troubles, right? And so these are 2 very parallel streams that you have to look at and say, all right, for one stream, you know, this is the AI agent's going to build on top of that. The other one, how do you decouple it from the work the others have done so that it can understand, have different fundamentals to say, yes, I know you're basing it on the past tickets. I may not necessarily look at the past tickets to do the evaluation. I will look at some fundamentals that my security engineers have told me that are like core to how we do things. things. They might see something different there. How do we handle that?

That makes a ton of sense, actually, the differences in what quality looks like. Can I ask an off-the-book question, which is, being Google, we have the benefit of having all the latest and greatest models happening just next door. How are you managing that question of whether you're using this Gemini or that Gemini or latest Gemini? How are you evaluating as the models march along?

That is a good question. I think When it comes to validating models and kind of understanding what we want to utilize, starting point for us is many times we utilize what's on Vertex, right? What's latest, greatest, what's out there, because that's the most stable, right? And so we start with that. When it comes to other models that are coming out, this is where we've gotten very structured in terms of how do we collect and curate and store our datasets, right?

Mm-hmm.

I think this is something that Sometimes I go out and help other teams with their AI projects, and their starting point is to say, well, yeah, we did a bunch of tests and we're good, right? It's fantastic. And if you don't have that rigor at the beginning in terms of how you've collected the data, how you stored it, how you've identified what's good, what isn't, that you have the appropriate snapshot, the second a new model comes out, it becomes very difficult for you to do that evaluation. The evaluation against other models, I think, is a little bit more simpler because you can go and then say— simpler relatively, of course. But if you have the right data, you can go run it against the model and say, well, how well did it perform? But if you don't have that starting point in terms of how have you collected the data, have you identified what works, what doesn't, have you defined what success looks like, you're going to get into a lot of trouble. And so I have seen other teams that I've worked with where a year later, 6 months later, 3 months later, a new model comes out and they're like, we have no way of evaluating it.

Oh, yeah.

Because you didn't get the foundations right. And that is one thing when people start getting into the world of agents that I kind of remind them to say, yes, it's AI and GenAI has made AI more accessible, but the fundamentals have not changed. If you do not approach this as you would a research experiment, gather your data, have your hypothesis, have your success go, no-goes, yes, you will definitely have some success earlier on, but keeping that momentum will get harder and harder.

It's perhaps not surprising, perhaps not disappointing, but yet again, It's really the fundamentals.

Yes.

Yeah.

Hmm.

So there's not a magic wand. There's, again, like everything, you have to do the basics right.

You have to do the basics right. And, you know, within security, I think this becomes even more complicated because you are dealing with very sensitive situations, sensitive playbooks, sensitive tickets. And how do you get access to those? How do you make sure you can process those? You don't necessarily have the volumes, right? I deal with some other teams where the volumes are massive, right?

Sure. Yeah. Fraud problems have much bigger volumes than security problems.

Yeah, yeah, tons. And with security, it's a much narrow surface. And so you also, want to be like, okay, I have a situation that appeared. How do I take that and then feed it back into my agent in a way that doesn't skew things, right? Because it's just one data point and the agent may look at that and be like, yep, that's the authoritative data point. Off I go.

But it's just one.

But it's just one.

And so let's talk about that risk. How are you managing the risk of the agent doing something wrong in an environment as critical as ours?

The interesting part on risk management here comes down to 2 things. One identity and 2 personas, actually.

Oh, okay.

Same one. Going back to—

That's such a PM answer.

It is a very— it very much so helps here, right? So when you look at maybe, you know, when we work with security engineers, many times they wear multiple hats.

Yes.

And that's perfectly fine. They get authorization to access certain things, do certain things based on the fact that they're a security engineer and they have many tasks that they have to do. As you start getting into agentic world, the parts that we start debating is, okay, but Who are you, right?

Mm-hmm.

What is your role exactly? Are you the malware investigation agent, right? Are you a security agent? Are you a privacy agent? Each one of these starts to have its own little sphere in terms of what it's supposed to do and its own sphere in terms of what it can do on its own, right? And so as we start looking at some of these agents, maybe a security engineer generally would be like, Great. You can have access to these systems and do these things by the nature of your security engineer. That same agent helping it, you'd say, well, no, actually, because this is a different persona. You're the— actually, you're the analyst, right? You're only allowed to do X, Y, and Z. Or you're the response agent. You are able to do this, where normally maybe one person would wear multiple hats. And so it is a bit of a PM answer, but I find that breaking down in terms of personas, be like, who really are you?

Mm-hmm.

And then the identity in terms of What do we expect from you helps you actually sketch out and say, these are the things you are allowed to do and not allowed to do. Otherwise, it gets very complicated.

That's itself like a fascinating answer because it suggests that the agent is forcing us to think critically about least privilege and the right privilege at the right time for the right task in a way that we didn't before because we said, oh, security engineer, we trust them.

I wouldn't say we didn't do it before, but before you had one person there that you could identify and handle. But they had multiple things they had to do, whereas now with the agent, because there's so much risk around that, because you don't want something going haywire, you have to really think about that and be like, well, no, actually, how do I really scope you down? And at the same time, consider that even if you would say the roles are equal, agents only have a certain view of the world right now.

Mm-hmm.

And that's something we have to take into consideration. So I may say, OK, fine. You are a malware security engineer human. You are a malware security engineer agent. But that human is generally going to have more visibility into what's going on. They're speaking with other security engineers. They're part of meetings, right? They see emails that are going. They're part of intelligence that comes in. That agent has been designed and works on a screenshot of the world, right? And they don't see anything beyond that. And so you also have to look at that and be like—

When is that going to change? When is the agent going to come to meetings?

I don't know. Maybe soon.

So I want to pick at something there, which is the role of humans and how it will change with agents. So, I mean, I'm being a little facetious asking when the agent's going to come to meetings, but also not. And so how's the role of the human security engineer, you think, going to change?

This is actually a very interesting question because I think the foundation comes in terms of, to start with, actually the interaction with the tools and the platform. What do I mean by that, right? I think a lot of, you know, especially maybe more on the ops side, but, you know, even later on, like, as detection engineering or other areas, the tooling is very much so designed and developed for humans by humans to kind of handle things, right?

At human scale.

Human scale, right? And so you may write something, but other people may look at it and be like, okay, fine, I understand how Joe talks about things, and so I can kind of interpret that. The big fundamental change that we're looking at now is to say, well, now you have an agent that cohabits a particular platform, right? They collaborate with you on something. because they only have a snapshot view of the world, how are you actually recording your information and the steps that you've taken in a way that machines can interpret and understand? Now, of course, GenAI and LLMs have made it easier for textual interpretations, but that becomes very important because if you put things in there that potentially mislead the security agent, right? Maybe you go down and you list a, you know, put a list of everything you'd like to do, right?

Mm-hmm.

And then later you say, oh, it was X. The agent may come in and be like, Oh, this is everything I have to do, and not realize that this was just a list that you had put down there to kind of track your progress, right? And so I think that interaction, being cognizant of the fact that you have this agent, which maybe it's like your intern that doesn't know everything, and how can I record things to help it understand, that's going to be, I think, a fundamental piece of the work that we're going.

So one of the things you said in there about Joe doing things one way or Papadopa doing things a different way, Do we have this actual hidden benefit of human beings being idiosyncratic and the agent tending towards a monoculture? Is there a risk that as we scale the involvement of agents, they'll have a particular and more narrow view of the world and we will miss out on the diversity of opinions that a diversity of human beings with a diversity of backgrounds can bring to the problem of security? That's also an off-the-book question, listeners, so don't hold it against Dominic if he tells me to get lost.

No. I think this comes down to the way we view how agents are going to contribute, at least in the near term. And the way I look at things is I want security engineers focused on the novel, the net new, the pushing the boundaries of our defenses. But at one point, that novel will become known. And that is the point that the security agents need to take over. And so I don't worry less as much about the diversity because the human is always going to be—

You still see the human doing the diversity and you see the agent doing the boring behind.

Exactly. And so that's the push because when we're looking at this, we really Like, if we want to stay ahead of threat actors, we want to be always on the cutting edge, pushing forward. Of course. And that's the best place that we want to, you know, the security engineers working. But once they figure things out, the agent will see this as it goes, right? And at one point, it will look at it and say, actually, now this is the pattern. Let me take this over, right? And the agent may even go further, Radar, and say, I don't need to do this anymore. I'm going to push this into automation systems, right, so that I don't have to handle it, or push it back further into preventative systems, right? And so it's that learning aspect. where looking at what humans have done, the advantage it can bring, right? Because it can have that view of the diversity of humans saying the same thing. Sure, sure, sure.

It can read more things than a human could.

Exactly. We have teams all over the world, right? And say, hey, actually, you know what? Sunnyvale, Sydney, New York, Zurich, you're all talking about something very similar. I think there's something new that you don't have to handle anymore. I can take it over. Where before, maybe we would've missed it.

Could you expand a little bit on something else you said in there about the agent pushing it into automation?

Yes.

Is there a world where, you know, we— We joke that the best engineer is a lazy engineer because they automate away the boring parts of their job. Is that true of the agent as well, then?

Maybe. I think that's part of it. Right. But the way I also look at this is it comes from our own desire. Right. We look at agents and we want them to fill a gap. Right. We say automation can handle a certain amount of something, but we can't get full autonomous because we're missing something. Agents can help us. As the human starts pushing more and pushing the boundaries of the fence, that's going to get more and more and more and more things that that agent has to handle. And if we have other systems that can handle it better, right, that are, you know, there for us and can do that at massive scale, we should look in terms of how we can actually push further left. And I think it's less about—

Are you teaching the agent how to write Endymion Hunts? I mean, is that what's happening here?

We're exploring many things in that direction, right? I think definitely, right? I think the parts that we're looking at is we really want the agent to come in, and look at the things that don't necessarily require security engineering skill sets, or it's something that we know very well and assist with that so that security engineers can handle more, right? For instance, how can we enable the security engineers to manage way more detections, right?

Right.

They can do X today. Could an agent handle some of the heavy lifting? Like, I imagine a world where you say, I've got these great ideas. You give it to the agent. The agent goes out, it fetches the information and compiles the reports. It says, hey, this is how I think you should update detection. Security engineer says, that actually looks great. Good. Tick, off it goes. But we really want to scale the security engineers across everything they do to say, how can we help you handle more? Similarly as automation. Automation helped unlock them, be able to cover more of Alpha, more of our customers. How can we do the same here? We're using agents to now actually unlock a brighter view that they can handle.

Okay.

Let me ask one more off-the-wall question, and we'll get back to the questions, and then we'll close. So listeners, I know this is fun, but we got to wrap up eventually. It's one thing for Tim or Mike to say to the team, hey, what did you guys do last month? What did you see last week? How do you get that kind of managerial finger feel out of the agent? How do you get a sense of what it's doing?

This is where I see things like quality agents coming in, right? So, you know, as I was mentioning before, we have the agent that's actually doing the work and kind of—

You have a manager agent.

And then you have a secondary agent that's actually verifying. And I think part of that flow is also looking at it and saying, OK, what have you seen? What are the patterns? Because it comes down to trust, right? I mean, if you're going to unlock agents and say, great, go handle everything, you also want to understand, what are you seeing? Is there something maybe here that the agent's been trained on a particular type of task, and you're looking at it and saying, oh, there's a different pattern that we haven't noticed here? And so actually utilizing that to provide a view into the world. And it kind of goes full circle to what I mentioned at the beginning, that distillation aspect, where you said summarization. I said, no, not necessarily. Can we leverage actually agents to help us distill this world and better understand it and provide some intelligence that then our leads can look at and make strategic decisions based on?

That makes a ton of sense. And I'm sorry I pushed back on distillation. I think everybody knows how much I enjoy the products of distillation. So that's great. I do want to pivot to our last question here, which is, as you've seen us deploy these, as you've seen things succeed, what are your worries on the flip side? What are you fearing that adversaries will do with agents, and how can we, you know, as Google, get ahead of that?

I, when I kind of look at things, I think part of what adversaries are looking for is just as we're automating, they're automating away themselves as well.

Yeah, they're just as lazy as we are.

Yeah, I attended DEF CON, and that was the exact actually comment that had come in DEF CON. It says, we are lazy. We want to automate as much of the attack as possible. And what I see actually with agents is, yes, automation, but B, also breathing new life into old attacks, right?

Oh, really?

Creating obfuscation, right? I mean, we see simple things like, for instance, even something like phishing attacks, right? Where before, maybe you'd have to have one person writing the emails, and maybe automation can handle this. But like, what happens if you can actually do this at bigger scale that is more deceptive and hiding the true attack, right?

Sure.

And I think that is part of the view that we start looking at is to say, all right, how are Adversary is going to start utilizing this to try and find cracks, right? Is that cracks in terms of noise or volume or things that we haven't seen before, right? We need agents on the other side to be able to handle just as much of that, right? And so I think that's—

So it's an arms race.

I wouldn't say it's an arms race in the sense that, you know, we want to keep up with how they are managing threats and how they're actually planning their threats and saying, do we have the adequate things to be able to defend against that?

Scary.

It can be, but it's exciting.

Okay, well, I have to do my least favourite part of the job, which is ask my traditional closing questions. And so, one, do you have recommended reading for our listeners so that we don't leave them empty-handed? And 2, do you have a tip to help people get started with a good foundation with LLMs or agents in their stack? Because we know people are doing it, but how do they do it with a good foundation?

Yeah, no, that makes sense. Starting point, honestly, The industry is moving so fast, is research papers. And the research papers are complex, but my recommendation actually is a lot of great YouTube channels that distill that into the basics, right?

Back to distillation.

Back to distillation. Everything's about that. That'd be my starting point. Also, just the foundations of how LLMs work. So a book that I recently read was LLMs from Scratch and LLMs in Production. Both are very good for you to understand how these things work, because if you're going to build agents, agents are built on LLMs. And so if you understand the foundations there, I think it helps you kind of move from there. In terms of kind of more concrete getting started, Google's ADK, Agent Development Kit, that is an excellent place to get started. We've had great feedback on this. It's a good way to start stitching things together. And so once you kind of understand the foundations of how LLMs work, and then you can apply them to agents, that'll get you well on your way.

Dominic, I love this. This has been so information dense, such a fun peek behind the curtain. Special shoutouts to various team members apparently occurred during this episode. So wonderful. Thank you so much for joining us today.

Thank you so much, and catch you next time.

And now we are at time. Thank you very much for listening, and of course for subscribing. Please subscribe so you can get new episodes piping hot. And if you love our content, please drop us a review on your platform of choice.

You can find this podcast on YouTube, Apple Podcasts, Spotify, or wherever you get your podcasts. Also, you can find us on our website, cloud.withgoogle.com/cloudsecurity/podcast.

You can argue with us on the Google Cloud Security Community site, googlecloudcommunity.com.

You can follow us on X, x.com/cloudsecpodcast. Tweet at us, email us, argue with us. And if we like or hate what we hear, we can invite you to the next episode.

See you on the next episode of the Cloud Security Podcast by Google.

Security. Security.