Dave, an AI security expert, reveals how Microsoft Copilot bypasses SharePoint and OneDrive permissions, letting any user access sensitive payroll data by asking the right question. He explains how to protect AI applications with agent constraints, policy-driven guardrails that limit what an AI can do and block prompt injection attacks.
Yeah, I mean, you know, some very famous examples where, you know, I'll pick Microsoft Copilot as an example. You know, they're obviously very early into this market. You know, arguably as well, a lot of customers get Copilot free of charge on an E5 license with them. So it's a, you know, it's a natural kind of testing ground for your first kind of iteration of your AI program in the business. Now, one famous example was that when you look at SharePoint or OneDrive, where you hold all of your content, And you have permissions on these folders. And so there are certain files that I can see that you can't see, for example, if we're in the same business. Now, one of the interesting aspects of a breach within a company was, well, payroll data, for example. I have access to that, but you don't. But the AI agents that Copilot were producing didn't make that distinction on the permissions. And so everyone could see everyone else's payroll data if they asked the right question of the LLM in that instance. And so, again, it wasn't an issue until someone discovered it. It's a good example of how new, exciting technologies maybe introduced some risk factors that could be quite serious. Payroll data can be quite sensitive in the wrong hands or with the wrong purview.
And so you've got risks for enterprises, right? If you're bringing in what could be a bit of a Trojan horse. What about if you're an entrepreneur and you're building out an app? How would it work, for example, with Aria versus working on N8n, for example, or Replit, or even Lovable? Maybe you could just help us to understand for the entrepreneur who's building something that they're going to use themselves or sell on. How does that work for them?
I mean, one of the other aspects, you know, as well as being able to orchestrate and build AI agents within our platform, We do have the side of it where you can build your security and your compliance and governance components as well. And so a good example might be like the Replit example. I would probably want to, if I was in that situation again with Replit, I'd have what we call an agent constraint in place. And what an agent constraint does is it looks at all the tools that the AI has access to do. So it has to be able to read databases and write to databases, for example, but maybe it doesn't need the capability to drop a database or truncate or delete a database. And so we could actually have a policy that says, well, we never want the agent to be able to do that. And if the AI, again, it's deterministic, it doesn't necessarily know, or you can't anticipate what it's gonna decide it wants to do with the decision of the information that's been given. But what I definitely don't want it to do is certain things with my database. So I could actually have a policy that says, well, This AI has access to do all of these capabilities, but I don't want it to be able to drop a table or delete a database. And if I put that policy in place, I've solved Replit. I've solved that issue altogether. And these guardrails are not just for tools, but they could be for things like maybe I've got a bot on my website. I don't want it to talk about certain things like my competitors. I don't want it to be prompt injected or be manipulated in any way that would give false information to the website visitor. Having guardrails in place means that I can monitor, track, and manage the language that goes in and also out of the LLM. So it gives a little bit more control than you otherwise would have had, had you had no guardrails in place.
I'm mindful of, with that, thinking about how when you have a new company and you have an office and you bring people in, you have policies, employee policies. And you have some guidance and some guidelines written down. But, but what you've really raised for me there, Dave, is the real risks that I'm actually kind of letting almost anybody into my new office and saying, do what you like kind of thing. We're working on this, but I haven't really got any security on the, on the doors. I haven't got any way to keep an eye on them when they're actually in there as well.
Mm-hmm.
2 people, infinite agents, 1 billion dollar idea #AIAgents #Startup #Productivity
