EP265 Beyond Shadow IT: Unsanctioned AI Agents Don't Just Talk, They Act!

Security. Hi there. Welcome to the Cloud Security Podcast by Google. Thanks for joining us today. Your hosts here, as ever, are myself, Tim Peacock, the Senior PM for, well, a bunch of stuff in Google PsychOps, and of course, Anton Chuvakin, a reformed analyst and now senior staff in Google Cloud's Office of the CISO. You can find and subscribe to this podcast wherever you get your podcasts, as well as at our website, cloud.withgoogle.com/cloudsecurity/podcast. If you want the video option, you can go to youtube.com/@cloudsecpodcast. If you like the show, don't forget to hit subscribe so you get our stuff delivered piping hot every Monday. And if you want to argue with us and the rest of our Cloud Security Podcast listeners, you can join us on the Google Cloud Security Community page. Anton, we are talking about shadow AI, shadow IT, things that go bump in the night, horrors untold.

Yes, but also shadow agents and citizen developers, if you want to make things a little bit more horrifying and you're just not scared enough. But it is the oddest part about this episode, by the way, is that we started by saying, yeah, there's a lot of FUD, and F in FUD stands for fear. And then in the middle, we were like, hey guys, we're not scared enough.

Hmm.

And at the end, we're like, actually, what we thought is optimistic, What we thought is pessimistic is optimistic and things are actually much worse. And then somebody said, no, no, no, they're actually much worse than that. So it is quite fascinating.

This episode did manage to get worse and worse without, I would say, going downhill, which is kind of interesting. It held together.

Well, not worse. It got better in terms of quality, but it got worse in terms of us scaring ourselves, actually.

Yes. But we did it without dirty bombs, which I'm very happy about. That was our last great piece of FUD, I think. And I think today we had more realistic Agreed. So maybe listeners, with apologies to Elie for me making fun of his dirty bombs for many years now, let's turn things over to today's guest. Today we're joined by Alistair Paterson, the CEO and co-founder of Harmonic Security. Alistair, thank you so much for joining us today. Delighted to have you. Your company focuses on helping enterprises secure generative AI. And I feel like there's maybe more FUD about generative AI than I've seen about maybe anything in my history in the security industry. And Anton, you've got another 80 years of security industry history on me, so you can tell me whether that FUD claim is fair or not. But could you walk us through a genuine example of what you've seen a customer struggle with and where it's not just FUD, it's actual loss or actual issue?

Yeah, yeah, great to be here, guys. And yeah, I've known Anton a little while. I think we can be pretty real here and get away from some of the sci-fi imaginary scenarios around AI and kind of what's really going on.

No robots rebellion today?

What? I mean, we can take this wherever you guys want, but yeah, I think the reality is AI security gets talked about with these huge market maps, 400 different subcategories and all this kind of stuff. But I think you've got to boil it down into you know, what is the problem that's being solved? So there's obviously threats from AI, like, you know, deepfakes and so on, not my area. There's use of AI for security, like SOC automation and stuff like that, not my area. Then there's helping you build your own AI securely, which is also not something we're focused on at Harmonix. So our entire focus has been on really the enterprise adoption of AI by the workforce, which I think is like super exciting. So there's a lot of stuff in there. And, you know, I know there's some pretty wild sort of sci-fi scenarios around even that, but I think, What we really see is the employees and the businesses pushing hard, of course, to try and accelerate adoption as fast as possible. And then security, legal, and compliance are picking up the pieces. Typically, step 1 is put a policy in place. Everyone's done that. No employee's ever read it. Step 2 is establish AI steering committee. Most people have done that now. It's just not doing a lot yet because it's pretty hard to even just get to grips with visibility into what's going on. And so rather than sort of focusing on the kind of out-there marketing pieces, we sort of put out some stats recently about what we've been seeing happening. And I think the reality is that most of the risk just comes around employees dumping sensitive data into personal accounts and other unauthorized apps, right? And so the sort of stuff we see there, though, is pretty significant. I mean, we've seen a ton of legal documentation and finances and things like that. There's a law firm we're working with who had a Intern sat in a bunch of meetings and was writing up the notes, and by writing them up, they got dumped into personal ChatGPT, right, as an example. And as these sort of personal editions get more and more sophisticated memories behind them, you know, that data is walking out to the next employer. And whether it's ChatGPT or it's Perplexity or it's Grok or whatever it might be, it is an issue, right? And we see, you know, across all of the prompts, we just published research into 22 million prompts that we saw across our customer base and file uploads into AI, and 25% roughly are sensitive. And of those, 16% go into personal accounts. So it's surprisingly high, right? And yeah, we can get into some more examples. I think one of the sort of business-critical stuff's more like IP related. We worked with one of the big designers and they had an enterprising employee fresh outta college dumping IP into some China-hosted sites.

Oh, what could go wrong?

Yeah.

And you know what? I guess, here is my guess. He was dumping it, he or she or they were dumping it in a Chinese AI because US one was banned.

You may well be right.

I am venturing a guess that they, because I've seen that risk decision when people ban a semi-seemingly legit but consumer-grade file sharing, employees find something really far outside of legitimacy that even the vendor doesn't know, the blocking vendor, and use that. I mean, I'm literally having flashbacks to the early cloud.

Ah, I can't have Dropbox, I can't have Box, I'll use Zandex file sharing.

No, no, no, Mail.ru.

Whoa.

No, not that. Mail.ru file sharing.

Yikes.

Because it wasn't blocked for some time. Yeah, it's also the real example from the bygone era. But this is— am I— why are we not scared about this enough? Sorry, this is a philosophical Alistair question. Why are we not? Sorry, what's the polite term for shitting our pants? Like, why are we not—

I think I started the episode by saying we have more FUD about this than anything else. I think we're scared enough, aren't we? Alistair, are we scared enough?

Yeah, let me sort of— what I see out there, there's really 4 types of companies, right? So there's companies that are basically trying to block all AI and they seem happy enough trying to block it. I mean, whether that works or not, we can debate. Second type is we're blocking AI, but we are under immense pressure to open it up, right? And we're terrified of the control challenge. And then type 3 is we're basically wide open and culturally I can't block it even though I want to in the security organization because I just can't, right? Everyone needs to be using these tools, but I'm terrified. And then type 4 is, hey, we're wide open, but we kind of don't care because we don't perceive a big risk here. And I'd say that most people are in one of the middle 2 buckets. And yeah, I think the sort of scary thing is that I think trying to block often doesn't work as we know, right? I had a really fun conversation the other day with a head of AI. who said, hey, Al, I'm head of AI, and this is a major insurance company in the US. I don't have access to ChatGPT. And so I said, well, what do you do?

What do you do?

I use this laptop. And he held up his personal laptop. And then he said, and so do my team. And it was just like, oh, wow. Well, I mean, what a great example of how blocking doesn't get you—

Yeah, I think I've heard that before. And I bet I have not heard it from Alistair. I mean, there are companies like that, where AI team uses shadow AI because AI is otherwise banned.

Exactly.

Yes.

Not just shadow AI, shadow IT with your own laptop.

Correct. It's shadow AI and shadow IT because you have to use shadow IT. Holy cow. I'm like, yeah, okay.

That's amazing.

Yeah. So to your point, Anton, I think there's a lot of, you know, if you try to block the employees, I don't think that tends to go well either. You've got to be leaning in here just with the right controls.

So this is reminding me of something from long ago in my life. I was a political science major, and one of the things I used to be very interested in was illicit economies. In illicit economies, there's this concept called the Iron Law of Prohibition, where if you prohibit a substance, people will traffic in more concentrated versions of it. So we prohibit alcohol during Prohibition, we get rum runners, which are easier to move quantities of rum than quantities of beer. We prohibit opioids, we get fentanyl, which is much more pure per gram. It feels like there's something similar here where as you ban access to software, people do more dangerous versions of that software. You ban good file shares, they go to bad file shares. You ban ChatGPT, they go to Shadow IT and Shadow AI at the same time.

And Chinese models. Yes. So I'm scared enough. Can we kind of do a tiny bit of positive. Like, at this point, I'm like shuddering here and thinking, is AI out to get me?

It is.

But I think that the common first thought that people have when they see this in their companies is, of course, they think of the boring word, boring G word, governance. And sometimes, I mean, I've dealt with this, we've written blogs about this, like governing shadow AI. But Alistair, what is your sort of advice, take? I don't know. What does AI governance mean in the context of shadow AI specifically? Because it's kind of ungoverned, right? But you want governance. So like, how do we do that? And also there's regulation coming, right? So there's governance mandated by regulation, but AI is in this case shadow. Explain the whole thing to us, please. Sorry for— I guess you're doing this as a business. You probably would know this, right? It is.

Yeah, it's a really fast-moving space, this one, isn't it? I mean, there's new things coming out every week. But I think the way organizations need to think about this is understanding the intent and the use cases behind the employees and how to enable them safely. So the starting point to that is visibility, right? It's understanding what is going on right now. You've got to get your policy in place, sure, and everyone's done that now. You know, hey, employees, don't put sensitive data into unauthorized personal apps, right? But actually getting the visibility into what's happening is key. And I think the important thing, I think security often, we sort of Think of the employees as kind of the bad guys in the scenario here, but most people, they're just trying to use this technology to boost their productivity, and they get used to doing it at home. Everyone's using it at home now, so why not at work? I think the huge benefits are there too. It's very interesting how enterprising employees can be as well. I'm hiring more than 20 people right now, so we've just been interviewing for a recruiter in-house, and I found a recruiter who was using Cursor off her own bat as part of the recruitment job that she was doing. So She'd built out her own workflows, and she was systematizing a lot of her approach to hiring people herself, right? Unknown to the employer or with some sort of tacit support. But I thought that was super interesting.

And yeah, why do you need an ATS when you can curate your own that works better for you because it's your own?

Yeah, exactly. You know, I think it's really interesting. You've probably seen the explosion over the weekend here around Claude bot and every, you know, something like 38,000 AI community members have been playing around with that to try and do their own things at home. I think it's just a natural extension of the more sort of forward-leaning folks is going to be wanting to use it at work. And then you've got to solve for the general case, which is all the other employees that are just dipping their toe, right? But I think the way to approach it, rather than just trying to cut this stuff off, is to say, okay, what, what are the use cases for each of my teams? Like, how do I get involved in enabling them? Where do we have gaps right now? So someone's using something, why are they using that? That's interesting. They're not using my sanction tool. Well, maybe that's because it's not fit for purpose, or maybe they just didn't know about it, or they weren't enabled on it. So I think it's taking a supportive partnership approach here is the way forward for the security organization. And then I think you're sort of then well positioned for any governance that comes in, right? I mean, we're waiting to see a little bit on aspects of EU AI Act and others, but to solve for that, I think that really comes down to understanding intent and use cases. So who's using AI for HR use cases and why, and what are the tools that we've given them to do that?

I think the use cases take just makes me really happy because it reminds me of my younger years of dealing with SIEM when I could have been like a broken record. Focus on use cases, focus on use cases, focus on use cases. And it seems like the focus on use cases, it's not a curse, it's the opposite, blessing. Focus on use cases blessing is coming back and, well, helping us give hopefully useful advice here. Sorry, I digress. Tim, it's your turn.

No, no, I appreciate the digression to broken record, Anton. I think that's great. You might have made a good PM actually with that broken record approach.

Just with that. Everything else, probably not.

No, everything else for sure not. So we kind of just talked about the governance side of it and the supportive partner side. I want to ask kind of the technical controls side of it. Like, if people are doing this, which we know they are because obviously they are because it's useful, what's the right technical controls to get the visibility and eventually the control? Is it network? Is it API? Is it endpoint? Is it identity-based? Like, how do we at least get the visibility, if not get the lockdown technically?

Yeah, great question. I think a lot of people looked at this space initially and thought, oh, you know, AI is a kind of a subcategory within my kind of CASB SASE, right? There's an AI category. That notion went away pretty fast, right? It isn't— I know I mentioned it earlier, but it isn't just about data being put into a handful of apps here. It's a far bigger picture now. And so, yeah, I mean, I think trying to imagine it as a category of SaaS is is sort of kind of woefully misleading at this point, because you've got everything. I mean, I mentioned Cursor and Windsurf earlier. I think this is the biggest shift I've seen in employee workforce kind of tooling in my lifetime, at least. I think we're probably similar vintage here where I grew up and it was all the Office suite and email that was basically work for employees. And then it became a bit of SaaS. And now you can get a whole chunk of your day done sat in one of these AI interfaces. You know, whether that's kicking off different agentic workflows through a, you know, an IDE that's enabled for that or some other special tool. There's a lot of different ways that you've got to look at this now. So yeah, I think the, you know, the rational starting and ending of a lot of this is going to be on the endpoint because I think that's where you have that rich interaction with the end user. You can understand a little bit more about their intent. So we started off looking at the browser. That was the kind of the center of gravity for AI for a long time. we now also, you know, have an MCP gateway and an endpoint agent because there's so much going on on the endpoint. And that is, I think, the key piece where you can see what's going on. You can also have supportive interactions with end users and coach them and nudge them appropriately instead of just breaking connections and blocking sessions and URLs and things like that. So I think it's really important to have that end-user context. And the more that we're using the more and more sophisticated interfaces, I mean, you look at Claude Cowork, what, 2 weeks old now? That has the power to be transformational, something like that. in the enterprise, you know, I think that's where a lot of enterprise value for AI is going to come from, is just automating some of the glue and the, just the sort of boring process stuff that people don't get excited about, but you start to apply AI effectively to that. And that's going to be, you know, outside of Silicon Valley, we're talking about the average companies here that are maybe a little less AI-centric. It's going to be huge, right? And so to do that, that side of this, well, I think you've got to be on the endpoint.

No, I think that part about the glue and the boring stuff is so, so spot on. I was just having a conversation with a PM colleague of mine. And I was laying out all these kind of amazing agentic things we could be doing in a part of our product. And my colleague says, yeah, that's great. I just got off the phone with an international Fortune 50 customer. They need the ability to mark which fields are required on this part of our workflow. I'm like, oh God. And it is incredible in this world where we can do so many AI superpowered security changes. We get dragged back into the past of business process almost immediately.

Yeah. Yeah. And isn't it interesting, like, for me, at least, the Fortune kind of 50s are slower on this stuff than a lot of the sort of 1,000-person, you know, tech, fintech-type companies that are leaning in on kind of on day one, they have less process already, they're sort of putting it all in place. There's a Fortune 50 that I spoke to 6 months ago, and they were in block mode, it's in my 4 boxes. They're now in—

In this year, in 2025?

Well, 2026. 2025, it was. They were in essentially block, like, we've got one AI, Copilot, put everything in Copilot, block everything else, was the stance until about 2 months ago, when they've come under enough pressure now that they're having to move into an adoption stance, right? And now we're in the middle of it and helping them do that. But I think there were more, like, the bigger companies were a little slower in this way, whereas I think sometimes they're the early adopters, And then it sort of filters down. I don't see that this time around. I think it's almost like the sort of 1,000 to 5,000-person companies are leaning in faster in some cases here than the big guys.

So that ban to embrace is the perfect transition for Anton's next question.

Yeah, I wanted to start it in a more kind of cynical, funny way by saying, if you talk to somebody from the Stone Age who's been under the stone for years and they still ban, But you just brought up an example of a Fortune 50 doing it in 2025. So I'm kind of a little— you shot my game a little bit, but okay. So if you talk to somebody who is still in the, we should just ban GenAI on the firewall or on the EDR or whatever they want to ban, on CASB, what's the compelling argument for moving from that ban to a managed secure adoption? And can you maybe share a story where it reduced risk for them? This is probably— this question alone would pay for the ticket to this podcast, which is free, by the way. Because to me, this is where a lot of companies struggle, is exactly that. It's like they can't build a case to their boss that they should switch from ban to managed adoption. Yeah.

I actually think the case here, the business case here is incredibly strong. I mean, in security, it often isn't, right? I was building threat intelligence with Digital Shadows, my last company before this, and threat intel was always a bit of a nice-to-have. I think AI is existential for business. And so the case to to govern it and enable it is just, we're finding it a lot easier to talk to people about this. So yeah, just to give you one anecdote, it's a Bay Area company, about 6,000 employees, very high IP business, lithium-ion battery tech and other things like that. And so they were having to be in pretty heavy block mode because they're pretty terrified of IP getting out. And so the business case for us, so after 90 days after rolling out Harmonic, they had a 72% reduction in sensitive data leaking out. but more importantly, a 300% boost in AI adoption by the employees, which that's the cool stat.

Oh, wow. So it is both security and adoption.

Exactly. Yeah. Because they could take down a bunch of the blocks, right? They were like forcing people to use Copilot and nothing else. And then suddenly it's like, okay, we can put something in the middle that gives us visibility and governance and coaches and nudges end users and doesn't just block them. And so then the end users start to lean in. And I think the opportunity here is to make the security team kind of the heroes of the AI steering committee and give them that that sort of visibility and ability to push the business forward more than it is just a challenge to block and stop bad things happening, which of course is part of the job. But yeah, I think that makes the business case around AI adoption in security much clearer than in many bits of security where risk is in the eye of the beholder a little bit, right?

I really like that. And I think it's also kind of both inspirational and almost like makes me nervous that I'm going to lose my cynicist, cynical security pro card. But it actually also makes sense to me. So this This is very good that you pointed this out.

I don't think you're in any danger of losing your cynic card, Anton. I wouldn't worry about that. Sorry, Al, I cut you off there. What were you going to say?

I was just going to say, I mean, the other argument here is just, well, hey, if you try to block it, we know that doesn't work anyway. I think if you would take a cynic's argument to what I just said, you could say, well, how much of that 300% was happening just on devices and in ways that weren't being tracked before, right? And now we've brought it into governance and control. Yeah, interesting time and not the time to be the guy saying no to everything.

So let's pick on that a little bit. What's the difference here between shadow IT and shadow AI? It sounds like one feeds the other or one necessitates the other. And do they have kind of the same solution? Do they have the same— should security leaders think about it in the same way? What's the relationship?

Yeah, I mean, there's a clear relationship, isn't there? It's the use of things by employees that you haven't authorized. But I just think this AI era is so much more compelling than before, right? I don't think— Like in prior areas, I didn't hear many employees campaigning to get Workday installed, right? That was the thing that the enterprise wanted to do for whatever reason. It's just a little different. I mean, maybe your Dropbox example's better, but I just think this time around it's like, you know, well, do I want to put some files somewhere that, I mean, that's kind of interesting, but do I want to have my job done for me by this really cool AI tool? Well, that's pretty compelling. So they, I think the temptation for people to use this stuff is absolutely compelling in ways that the prior eras were not. And then I'd say, I mean, it's, we talk about it as if there's AI and not AI, but it's kind of everything's AI now anyway, that every existing SaaS platform is building it in. There's now AI browsers, there's AI-first IDEs. And, you know, I think it's just kind of, I think about it more as that was the prior era. And then now this is the current era. We were pre-AI, now it's post-AI, everything's got AI in it, and we got to deal with that. So I sort of think about it as, I don't think they're two distinct things anymore. I think it's just We have to deal with unmanaged use of stuff, which is mostly AI-based now and is going to continue to be the case. So yeah, I just think this is more important. It's happening faster. It's a bigger temptation for employees. And we've got to be on the right side of this in security.

Yeah. I think this is a good balance of scary and kind of optimistic slash inspirational. But I want to probably— I feel like the scary is not We need a little bit more scary. I feel like this dish, to cook it really well, we need to add a tiny bit more fear. And I feel like in this case, it's the shadow agents. I've been, well, it's not a big secret, but I'm doing an RSA presentation on shadow agents specifically. It's kind of interesting that I'm realizing that a lot of things that look scary just aren't scary enough because people give all sorts of powers. So how are you thinking about shadow agents? Specifically, the scenario that I have in mind, one of them, is that it's a consumer-grade AI agent, fairly general purpose, that employee voluntarily connects to corporate systems.

Yes.

Like that— walk me through that and tell me why I should not just move to New Zealand and dig a bunker.

Yeah, I think we think about this one a lot. I mean, I think you look at these AI systems, and they only become really powerful when they're given enough context, right? I think that's directionally where we're headed is how do we surface more corporate context and get it into these AI tools to make them better. And so I think, you know, just sort of simple examples around this that we've seen with the MCP gateway is people connecting through to CRM systems and pulling through customer data that then becomes essentially uncontrolled and pushed into other tools and devices in order to automate some pretty interesting workflows. And so, you know, I think that's probably one of the most immediate risks is that proliferation of MCP adoption that's essentially going unchecked in a lot of companies, and we don't want to stop it. I think it's a great interface, a great way of speeding things up. But again, we've got to bring some governance around this. It's sort of the latest chapter. It wasn't really a thing a year ago. It only got invented just over a year ago. Now we have a product to deal with it because it's become such a big deal. But I think that this is mostly going to be employees trying to do things better and faster. But The other thing that I hear that I think is closely related is this idea that we're going to enter this utopia of sort of citizen developers and companies building all their own SaaS products, just vibe coding away everything that they need to get the job done. And I don't see a great outcome there because, well, one, I think it's kind of the obvious stuff of, you know, systems of record and so on. You need certain control and governance around it. And I think there's always a place for that. But I think the bigger one is the citizen developer thing. Scares me a little bit because you've got little teams building their own tools for their own purposes. There's a lot of that going on at the moment that's left undetected, and then employees leave, it becomes unmaintained, who knows what it's connected to, it becomes a big mess. And we've seen that before in different flavors. I think it's just, you know, it's history repeating itself for AI now with what's happening with these AI workflow tools.

Okay, I think that AI agents, consumer-grade shadow AI agents, was my scary scary monster number one, but you had to cattle prod me and remind me about the citizen developers recoding all enterprise SaaS by hand using agents. Not by hand, using agents. And I think these two are equally horrifying in terms of security. So I think these are like, it's like probably we should probably do a poll after this episode, like which one scares you more, dear CISO? Okay, Tim. Yeah, sorry. I just randomized you.

Dear listener, what is the scary monster here? Is it the one with fangs or the one with claws? So I hate to do this, and listeners, if you're watching the video, you'll see that I actually managed to migrate meeting rooms in the office to ask our closing question. One, given the pace of change here, what's the advice you would give a CISO to just even keep up with this stuff, let alone get their arms around it? And then 2, do you have recommended reading for our listeners?

Yeah, I think rather than this being— I mean, it is a little terrifying, don't get me wrong. I mean, I think I wake up every week and it's, you know, what's been released? Okay, how are we going to secure that? So sure, that's the thing. But I think there's a huge opportunity here for security teams to become strategic partners to the business that wasn't there before. You know, you walked into a room and you talked about your DLP program 2 years ago and everyone, you know, probably went into snooze mode pretty fast in the leadership team. This time around, it's like, okay, yeah, we can come in and enable this AI adoption wave. And I think it starts by understanding the use cases that each business partner of yours has and meeting them where they are, right? Understanding what they're trying to do and then finding ways of putting the right governance and control around it that everyone feels good about it. So I think that's the, you know, being that enabler. I mentioned earlier the 300% boost in AI adoption stat. You know, if you can be a security team with that type of enablement power behind, then the business is going to take that really seriously at sort of, you know, CEO level. And I think that's what we're seeing as well. The security team's often maybe the team that understands AI better than most, and sometimes better than all. And the CISO therefore becomes a bit of a thought partner for the business. So I think that's the opportunity for this era, is leaning into that, making sure you're the enabler that can make this stuff happen and that you understand it better than anybody else. So yeah, I think huge opportunity.

Yeah, I think that's a great answer. Do you have recommended reading? Can't be Anton's blog.

But it can be your own research, of course, because we're going to mention the data points you highlighted before.

We actually did. So yeah, harmonic.security, we just published some new research that has analysis of more than 22 million prompts that have gone in and different files that have been uploaded by a huge range of different types of corporate into different AI sites. So some really, I think getting beyond the sort of scare stories and the stuff that makes for good listening and good blogs, it's just a good statistical breakdown of where the risks are. So if you want to take a practical approach that's evidence-backed, I think it's one of the few datasets that is published around that.

Who wants to do that? This is the era of vibes, baby.

That's right.

All right, well, Alistair, that's an amazing resource. This has been such a fun conversation. Thank you so much for having joined us today.

Always a pleasure. Thanks.

And now we are at time. Thank you very much for listening and of course for subscribing. Please subscribe so you can get new episodes piping hot. And if you love our content, Please drop us a review on your platform of choice.

You can find this podcast on YouTube, Apple Podcasts, Spotify, or wherever you get your podcasts. Also, you can find us on our website, cloud.withgoogle.com/cloudsecurity/podcast. You can argue with us on the Google Cloud Security Community site, googlecloudcommunity.com.

You can follow us on X, x.com/cloudsecpodcast. Tweet at us, email us, argue with us. And if we like or hate what we hear, We can invite you to the next episode.

See you on the next episode of the Cloud Security Podcast by Google.