EP260 The Agentic IAM Trainwreck: Why Your Bots Need Better Permissions Than Your Admins

Security.

Hi there. Welcome to the Cloud Security Podcast by Google. Thanks for joining us today. Your hosts here are myself, Tim Peacock, a senior PM on Google SecOps, and of course, Anton Chuvakin, a reformed analyst and senior staff in Google Cloud's Office of the CISO. You can find and subscribe to this podcast wherever you get your podcasts, as well as at our website, cloud.withgoogle.com/cloudsecurity/podcast. If you enjoy our content and want it delivered to you piping hot every Monday, please do hit that subscribe button. You can follow the show, argue with us and the rest of our Cloud Security Podcast listeners on the Google Cloud Security Community page. Anton, we had, you know, there've been so many episodes where you've said, you know, that insight pays for the ticket right there. And this episode you say that, Oh, I don't know, 3, 4 minutes in, and it was about modelling how things are structured, which I think is interesting. Usually you wait until it's like 15, 20 minutes in and listeners have to listen to the whole thing, but today they get it right up front.

Yeah, but I don't think I'll wait for 15 minutes. I think I just blurt out things without thinking, admittedly, in that particular case. Not every time, hopefully. But I think that we are trying to solve a lot of problems with the CK. securing agents and securing LLMs. And I felt like this episode was a broad exploration of a topic with some gems. So I don't want to make excuses for ourselves. We are always awesome, right, Tim? We are always awesome.

No, not me. I'm rarely awesome.

Okay, I'm always awesome. Fine. And Tim is usually awesome.

Occasionally awesome.

No, more than occasionally. So frequently awesome.

Frequently awesome.

But this episode was an attempt to explore Broadly. So some of you may think, hey, it's broad, but it has also a little bit of deep elements, kind of little gems in the broad exploration of securing AI and securing agents. For example, I think at minute 17 or thereabouts, we solved the agentic IAM. Did we?

At precisely minute 17, listeners.

No, no, no, no. I'm rounding things up and down and sideways too.

Oh, good. Well, one thing to look for, listeners, I guess before I turn things over, is thinking about how to separate and reason about applications built with LLMs. And so with that teaser, let's turn things over to today's guest. Today we are joined by Vishwas Manohar, the founder of Precise AI. Vishwas, thank you for joining us today. We want to talk about agent security. And so how is agent security different from, you know, regular old— now that it's old— LLM security?

Yeah, really good question. Right. So if I was to look at it from a, you know, the older sort of the way applications are built, you have a 3-tier application, right? You've got your database tier, you've got your application tier, and you've got your presentation layer. In the traditional world, the database was, you know, it could be SQL, MySQL, whatever else, Oracle, right? The application tier could be microservices, could be something else. And the presentation layer was the UI. That application stack is changing right now in the AI world. The database is replaced by the LLM. LLM is like the database that can give you data that's in it, but with all the hallucinations that it has, it can give you data that's not necessarily in it either, that you've never trained it with. It can create new data and sort of give you the data from there. The application tier now is actually agentic. So when you talk about agentic, it's about applications that do not just look at data and present it to the user. It's actually now connecting to various tools at runtime. Doing actions that it has never done, creating new logic at runtime and being able to perform certain activities and actions and connect with tools and so on. The presentation layer is, again, it used to be very UI-centric, very data-centric in the past. The new UI layer is going to be very, very natural language-centric. And so when you're talking about LLMs and agentic security and how they're different, you have to think of it exactly equivalent to to how database security is different from application security. Applications have totally different concerns, databases are totally different, right? And so, that's how I—

Sorry for stopping you.

That's great.

It's actually kind of really smart. I really like it. I mean, maybe for Tim, it's a little '90s, but it's also really crisp. Like, it has database security and application security. I've met enough people who think they're together, and they, like, say, NetSec is on the other side. But they're not really together. They're different enough in many ways. So thank you for this metaphor. It's already, like, I think that metaphor already pays for the price of the ticket.

Great. Absolutely.

Yeah.

And then from when we are thinking about application security, we have to think about actions and actions that it's doing that are not necessarily right. That's where you're looking at agentics. But if you're looking at LLMs, LLM is a component, right? It could be hallucinating. It may not be hallucinating. Just like in a database, a database could go down, could go up. When I'm building an application, I have to make sure to build distributed applications such that even if one component goes down, goes bad, does something wrong, the application itself doesn't get affected. Same thing if LLMs are hallucinating. I have to build my applications in a way that even if individual LLMs hallucinate or individual components hallucinate or individual components do not work properly, the application does not get affected. And that's how I treat this whole sort of thing as I look at it.

I really liked it. And I think that it explained the— I mean, I guess calling LLM security traditional is already a kind of a big stretch. But just a few times, actually a couple of weeks ago when we were at the CISO event we run, a lot of people said that agents are coming and they're coming to companies that are barely managing just chatbot security and the fact that LLM was added. And it's like the new, new thing is kind of crashing into people who are barely handling the new thing, and then suddenly there's a new, new thing. And then one of my colleagues in Office of the CISO produced a scary and frankly terrifying thought. Think about this. This is November 25th. Go to November 26th, and there's this new, new, new thing is coming, and you are like totally underwater with LLM, but barely managing. Agents still freak you out, and then suddenly, plop, there's this new thing lands.

Yeah.

I mean, it's stressful.

Yeah. I mean, we are in a fast-changing world and, you know, it's, things are going to be changing fast. And so we have to adapt to that world. I mean, to be able to play in it. See, it's like AI is like a tool, right? And we cannot go to a gunfight with a machete, right? As an example, we have to sort of use those tools and we have to be ready to adopt the tools because things are changing really fast and we have to be able to, you know, be ready with it. I think that's how I look at it.

So I want to ask about the threats and risks. And I feel like agents are new enough so that the wide use remains to be a kind of a contested topic. If you ask some people, they report many POCs, they report agents in prod. My former colleagues at Gartner have a lot of research about the state of agentic adoption in the enterprise. But then of course, there are reports about like pretty high percentage of failures of projects. So like, it's all very uncertain. But is this uncertainty going to dissolve and we'll see the top, top, top risks? Because to me, if somebody wakes me up at 3 AM and says, Anton, agentic risks, I think of data loss, I think of prompt injection and some rogue action. And then I'll pretty much just call it a day. Maybe I'll remember DoS. I usually remember DoS. But you are working on a top 10 risks list, right? Well, you have one online. So, what was the motivation behind it? And why not just wait for people to blow themselves up? and then count what the problems are? Why do it proactively? Sorry for a morbid metaphor.

Absolutely. I think it comes back to how fast the world is evolving. I have been part of standards bodies before, so I've created— I'm an author of some of the IPsec standards. You know, it took 7 years to create. Interestingly, I found my name in one of the TCP standards. So the core TCP RFC has my name. I think I contributed to it in 2008. And the RFC actually came out in 2022, '23, I don't know when, right? So standards take a really long time to build.

That speed, hyperspeed. Exactly.

We are in a world that's changing really fast. And so standards and the way standards are built are also very different, right? In the olden days, standards were like every bit and byte was totally constrained. And, you know, we knew exactly how things would be for the next 20 years, 10 years. And that's why standards had to be very, very stable and static. The world we are headed in, or the world we are in, things are changing so fast. And so the standards are not like a hard limitation. It's more like a line in the sand that can be changed, updated, upgraded. And so when I'm talking about OWASP standards for agentic AI, it's more about— or just top 10 standards or top 10 issues for agentic AI. It's more about giving people guidance on how things are right now, the state of affairs right now, the use cases that we are seeing. Of course, it's bound to change. But that's how we have to treat standards right now. We cannot treat it like the way things were in the past. That's how I look at it.

I mean, people bring up compliance regulations, not even industry standards, and they say, well, if we are adopting AI for business, and of course, adopting AI for security, like, reminds me of the episode we recorded earlier today with Tim, weirdly enough, like, regulations are way slower than industry bodies. Sometimes regulations wait for industry bodies to settle. And then take their own deliberative time. So, I'm sorry for, like, going off the script a little bit, but it's kind of, like, a little hopeless in the world we live with AI to rely on the old standards and regulations, right? Or is it still a reality for many people and organizations?

I would say, yes, it may not be very useful to rely on old standards and regulations that sort of do not meet the current needs, because that has happened with a lot of older standards. You see, they became more of a, like, a a tick mark than actual security standards. And that's what happens because the world has moved and the standards do not move. But the way to look at it from my perspective, when I'm looking at regulations, regulations are meant to be slow. We don't want regulations to change every 3 months, 6 months. It creates a lot of pain in the business environment. I do not know what to design for, what to build for, and so on. And so regulations are meant to be slow. The way I look at how regulations have to be from here Regulation should not be broad and thin, which is how regulations have generally been. I think regulation needs to be narrow and deep, right? Very specific, but very deep. And I think the way to do regulations have to change to make them relevant as we move forward. Right now, it's like you need to get a SOC 2 standard. Nearly every company has a SOC 2 standard and you have your latest ISO 27001, but they're all tick mark items. They're not real standards because, or regulations because of the fact that The current reality and the regulations are not sort of in sync with each other.

So I want to come back to agents for a minute now that we've allowed Anton to make a regulatory detour. Allowing Anton to make detours, listeners, is one of the interesting things about hosting a show with him is sometimes Anton will take a detour and I'm like, God, why are you doing that? And then it turns out to be fascinating. And other times Anton takes a detour and all I think is, God, why are you doing that? But to bring us back—

But you can't kick me under the table because we're not in the same room, unfortunately.

Unfortunately, listeners, I have no means of kicking Anton under the table. But I want to come back to agents and talk about IAM. How are we going to solve— and it's okay if this is an unsolved problem because I think it might be— but how do you think we're going to solve agents plus IAM? What does that look like?

And by the way, before you go answer, I just want to set it a little bit, and not a detour promise. I met 2 groups of people. Some people think that agent is essentially like an employee, almost like a human employee, and the other half thinks it's a workload. And the challenge isn't that there are 2 groups. The challenge is each of the 2 believes that they're the only source of truth, that nobody sane can possibly think that, and they quote the opposite position. But both exist. So that's my kind of frame setter here.

Got it. Got it. So from my perspective, agents can take human— if you are sort of running agents on your machines and they're doing tasks for you, they're going to take my role, get my authorization, and do things. That's how it's happened. That's how it'll happen. And the way I look at it is agents and IAM is similar to microservices and identities in certain ways, because services come up, go down, they connect with other services, they have temporary, they're ephemeral, they sort of come up for a very brief period of time, they can take certain roles, responsibilities, run on different systems, multiple of them can come up on multiple systems, and so on. So I feel they're very similar in those aspects. However, because microservices generally run in the backend systems, yes, if agents are running in the backend system, they can use identities from these backend systems like SPIFFE and SPIRE and use similar models. But when agents are running on behalf of me, I do not think you can have those kind of models. You have to have a way that these agents actually take my role. But the authorization, I feel, has to be constrained because if an agent is taking my role for a particular task, it's doing say, a build for me, it should not be allowed to do, again, suddenly, you know, use my credit card and do something else. So it can take my identity, but only for very limited authorization. And I think that's an angle that we have to sort of look at. I know there are papers out there, but this is like Tim was mentioning, this is an unsolved problem in my view.

Yeah.

As it stands. And there are similar problems in, like I said, in the microservices space, in the cloud world too. I trigger or check in a code on GitHub. GitHub triggers, with GitOps triggers Jenkins. Jenkins does a build, something else deploys something in the cloud. The cloud teams think resources deployed by a service identity, but there is a whole chain of command and all these chain of identity that are related is really hard to trace and track. It's not a unique problem. It's not Sort of, I would say it's not a problem that's not been seen in the non-agentic world. It just gets magnified a lot more in the agentic world. And that's how I see it.

I like it. And I think that it answers the question because right now I'm observing a lot of debates about this and people either claim to have an answer or whatever else. But ultimately, you're right. I think that a human for some of the roles in many cases, but authorization, like constraining it like a workload. Because for a second, when you went to microservices analogy, I thought, uh-oh, he's from the agent to the workload camp. But I think that it's the analogy rather than the agent is a microservice. I think that makes sense. So, I wanna switch back now that we quickly solved agentic IAM. Haha.

Haha.

I wanna switch back to risks here. Top 10 is great, but is there— I'm gonna just use a straight 3 AM test. Somebody wakes you up at 3 AM and says, name one risk that— well, wait, doesn't keep you up at night because you're sleeping. So, name one— sorry, I lost in my metaphor.

So, name one agentic risk that scares you but does not keep So I think the biggest agentic risk is when agent does something that it's not authorized to do, and that agent has been given a lot of different permissions. I think that's the biggest risk. I think some of those risks are very easy to deal with from a user perspective. So we build our own agents. We do not— our agent, because we use models that have the world's data, it could probably, if I'm connecting with the cloud, it could call cloud APIs even if I did not give it the right API access, right? Because it has the data and so on. But what we make sure is when we are sort of working with agents, we do not give it direct access to the cloud, as an example, because agent knows about the cloud. If I give him access to the cloud, it could do things that the cloud can also do, do things in the cloud because there's public information about how cloud APIs work and it could call those APIs. What we do is we've got a proxy layer for these agents and I only give it certain APIs and certain access only there. So what that means is an agent cannot necessarily call my cloud or call any public APIs directly, and it does things within the constraints of what I want it to do. And that's how we've solved this problem of agents doing things that it should not be doing, and which I think is the biggest fear for companies when they use agents. Again, the issue has to be, you know, you cannot allow a god agent to be there. That is an agent that could do almost anything.

Yes, correct.

You have to make sure that you're only giving the agent certain permissions certain rights, certain APIs, and make sure that it doesn't get access to APIs that can be called without— like I said, like I give an example of APIs for cloud, right? Because if I give it access to cloud, it could do a lot more, right? Assuming the authorization is there. So I don't want to give those kinds of authorizations or those kinds of permissions. And that's how I sort of treat it. But the biggest fear for me is always that, you know, the agent does something that I'm not expecting it to do and it sort of goes I think it's the rogue action bit, right?

That we, in our safe.risks, we kind of call it a rogue action. And I'm just in agreement. It feels like it's a— Vishwas's answer is both obvious but also insightful.

Hmm.

Because, like, yeah, I have this machine that does something. The scary part is it does something I don't want. So it feels obvious. But it's also insightful because people use agents or at least trying to use agents without sufficient safeguards against that exact thing. So it's like, whoa, obvious risk. Whoa, no safeguards. That's true.

I'll say one thing. The reason I told you don't give agent access to things that are publicly known is because agents, if you think of the sort of agents use LLMs for doing all the logic and so on, or doing actions. LLM is trained with all the world's knowledge. So if you were to see an agent, it's actually the average knowledge, you know, it would work like an average human with the average knowledge. So, you know, if I tell an agent to do something, it has knowledge of it, it might act on certain things even though I have not told it to do. Because say, think of it, 2 people, Anton tells an agent to say, you know, don't do X, and I tell an agent to do X. An agent is going to behave with Anton saying, hey, agent, because Anton, you've told me not to do X, I'm not doing X. While for me, it'll do X because that's how humans behave. Think of it. I have 2 managers or 2 people who are telling me conflicting things. I'm going to sort of behave like this average human, right? And agents are trained with the world, real-world data. So it's bound to do things that we do not want it to do because that's exactly how humans work, right? And so if you do not give it access to public data or do not let it use its public data and only give access to proprietary APIs through which you can again access public data, it gets a lot constrained.

I want to go back to something you were saying a minute ago about cloud APIs and ask a very cloud question, given that this is still, despite our discussion of renaming, the Cloud Security Podcast by Google. Is there a shared responsibility metaphor we might draw here? Is there a responsibility between the agent host and the agent user, the agent developer and the agent user? Is there Could we start to think about splitting that out a bit?

Absolutely. And in fact, Anton and I have been part of some of those meetings with the— in COSAI. I think Google has contributed a lot of the work that they've done on their sort of governance frameworks to COSAI. And as part of COSAI, we've actually been working on some of the shared responsibility models. And the models, while they look very different, are very similar in certain perspectives to the cloud shared responsibility model. There's a provider of the model, there's a provider of the user of the model. There is a provider of the application, user of the application. It's exactly the same. Just like in the cloud, there's a shared responsibility model. The provider tells you that, hey, I'm going to be owning this infrastructure. If it's an IaaS infrastructure or it's a PaaS infrastructure or SaaS infrastructure, with SaaS, we've got very clear responsibilities. The provider is going to be responsible for patching, for the software upgrades, updates. If it's an IaaS and you're running software on top, you are responsible for the software. The OS may be managed by the cloud provider. Similarly, in the agent, when an agent is provided, or agent, or maybe even a model is provided to somebody, there's going to be a shared responsibility model. The agent or the model provider is going to tell you what the model does, what constraints there are, how they're going to update it, how they're going to upgrade it. And a user can now build applications on top of the model or the agent and do different things in different ways. The constraints that the— as long as the constraint that the provider has provided are not violated, the provider is not going to be responsible for things that the user is using it for. And it sort of breaks at the application layer or at the user layer. For things that are sort of happening at the application layer or the agent layer, if things sort of break, it's going to be the responsibility of the provider, the agent provider. So it's very similar in my view. The responsibilities are very similar, just like in the cloud shared responsibility model. The only thing, the providers and the users and the way things are sort of placed becomes slightly different because now you've got a model that could be coming from a third party, you're hosting it, moving around.

And of course, the tuners and the hosters are there. So there are a few more dimensions. Yeah. I mean, I like your answer because you are not trying to over-engineer it. Again, it's not a jab against CoSI where we do work together on exactly that effort. But occasionally, I do get the vibe of the old standard bodies energy where people want to overengineer for every edge case, like, for literally every edge case. And there are some edge cases that are simple. Foundational model from vendor A, tuned for industry by vendor B, hosted by vendor C, included in an application by vendor D. That's pretty simple, by the way, as far as this responsibility. But there are also, like, way more strange things when there are consumers and businesses. And then I kind of, like, I wanna hide under my desk at this point because I'm not sure how this can be decoupled in a clear, something blows up, who is at fault? Because ultimately, it's gonna be a short rant. Ultimately, to me, the use case for shared responsibility is this. Something blows up, who is left holding the bag? It can't be nobody. It can't be, I thought it's X, but it's really Y, or worse, I thought it's X, but it's me, which, again, may happen. So, to me, if it works, It's a good model. If it doesn't, back to the drawing board, right?

Love it. Yeah, I think you sort of said it very subtly and very well.

But you think it's harder or easier than the cloud shared responsibility?

I think it's slightly harder because there are more, you know, sets of providers, but I think it's very similar. You could map it similarly, right? Just like a database. Like I said, if you look at it from a database tier, the app tier, the presentation layer, it's similar, right? Model is the LLM. You've got LLMs that could be hosted by a provider in your model garden, which is what Google does in certain cases. You could run your own models with, or, you know, a PaaS model where it's somewhere in the middle. So I think it's very similar. Yes, because of more sort of intricacies, there are slightly more complicated, but it's very similar and can be mapped very similarly.

So I wanna shift gears a little bit, even though shared responsibility can never really be shifted out of. We see when we talk about securing agents and top risks and rogue actions, there's still a question of adoption. So the question to me comes up as people are experimenting, but there's sometimes reports of little value from agents, or sometimes there is reports of risks. So what do you think is the biggest challenge of secure agent adoption today?

Yeah, I think the biggest challenge for adoption of agents is not the action. See, we've always been talking about agents can do action, and so they're different. They can connect to tools and perform activities or action that were never possible. My take is that's not the biggest challenge. The biggest challenge is trust. If I do an action on my environment, what is the downstream impact of it? If you look at a Stanford report, I think from the SALT Labs or something, it talks about the biggest fear enterprises have in adopting agents is not about actions, it's about the trust. What if I enable it and it does something that it's not supposed to do? What if I enable it and I don't know the downstream impacts of it and it does something? And so this trust is sort of the biggest fear for actions. And as long as somebody can build a layer of trust, so either with the human in the middle or not, where before an agent takes action, you know exactly what can happen, what will be affected, what is the downstream impact for business, you can take the action. Or in certain cases, when there's a risk and the probability is high that the risk could be big, you could move to a human action. But what is required is this layer of sort of trust that can tell you doing this action is less risk because, as an example, this is a sandbox environment, so you can take this action immediately versus, hey, this is a production environment running very critical data, don't take this action right now.

Right.

I mean, and I'm simplifying it, but yeah, that's it.

Of course. So, but sometimes when I hear trust, I feel like people cover a lot of hard problems with trust umbrella and say, look, trust is needed, but it doesn't mean that cybersecurity controls need to be there for AI, for traditional security. You need to have privacy, you need to have compliance, you need to have other things. So, when you say you need trust, can you clarify it in a more actionable way? And then we start to close the show because it's almost 30 minutes.

Sure.

So, let's talk about it in terms of cloud itself. When I'm saying trust, so think of a vulnerability you get, cloud security. Let's say you get a cloud security vulnerability. The processes of fixing a vulnerability in every company are very different, right? And there are different steps. When you get a vulnerability, you have to route the thing to the right developer. You have to know how to fix it. You should know whether the patch is right. You have to test the patch. You have to do multiple steps to sort of fix the vulnerability. When I'm saying trust, what I'm saying is, if you get a vulnerability and an agent can do all these actions, does it know the worst-case impact of this change? What will happen if I change this? Do I have all the context of this environment? Do I have all the context of what has happened to this resource before so that I'm not doing something without knowing what all this thing will impact? So in the cloud world, if I'm fixing a vulnerability and maybe I have to restart a machine to fix the vulnerability, what if I patch it and I restart it? What happens? What else gets affected? What do I need to do? If all that data is there, you now can have a layer of trust Which now a human could use to sort of enable that action, or an agent could do it itself. That's how I sort of treat trust when I'm saying trust. You're right, trust could be a very broad term and could be used very differently. And I'll add one last term because you started saying, you know, you're closing it. My name is Vishwas, right? And Vishwas actually means trust in Hindi or Sanskrit.

Does it?

Very relevant for this.

Whoa, now that's a bomb to drop at the end of the show.

That is that. Well, okay. So listeners, I trust you have found this to be as engaging as we have, but I have to ask our traditional closing questions. One, do you have a recommended piece of advice to help people better reason about agent security? And then 2, do you have recommended reading? Some kind of resource, could be anything from Mary Shelley to Mary Oliver, just not Anton's blog.

For the link, I would say you can search for Precise and agent top 10 risks, and you'll get a sort of a a GitHub link for it. So you can get that. That's one way to look at it. And for the risks and how to think about it, we are actually headed to an agentic world. I think we are very early in the cycle. We are already looking at AI being used in very different ways. So AI being used by humans to cheat humans, which could be things like malicious emails or spear phishing, but now with AI, where it becomes very contextual for you. You have AI cheating humans in certain ways by itself. You can have humans cheating AI in certain ways, which is what you'd call prompt injections. You're also going to have attacks like AI cheating AI, where agents are trying to cheat AI because agents are communicating with each other. And so we are very early in this world. We are seeing very early attacks, like I gave you the example of spear phishing or prompt injections or others.

Right.

I think this is a really broad world. As we move forward, we're gonna have a lot more attacks, and a lot of them are going to be AI trying to cheat AI into doing things that it should not be doing. So it's gonna be a lot more interesting than what we've actually imagined or what we actually thought. And I'm sure it's gonna be an exciting world for security as we move forward.

Well, Vishwas, that is certainly an interesting future to predict. We keep having more and more interesting attacks of AI, including on AI. So that's a great note to end on. Thank you so much for joining us today.

Hey, thank you, Tim, and thank you, Anton.

And now we are at time. Thank you very much for listening and of course for subscribing. Please subscribe so you can get new episodes piping hot. And if you love our content, please drop us a review on your platform of choice.

You can find this podcast on YouTube, Apple Podcasts, Spotify, or wherever you get your podcasts. Also, you can find us on our website, cloud.withgoogle.com/cloudsecurity/podcast. You can argue with us on the Google Cloud Security Community site, googlecloudcommunity.com.

You can follow us on X, x.com/cloudsecpodcast. Tweet at us, email us, argue with us. And if we like or hate what we hear, we can invite you to the next episode.

See you on the next episode of the Cloud Security Podcast by Google.