Agentic AI in Cybersecurity

What is the difference between generative AI and agentic AI and autonomous AI? Give me some— what are we talking here? What's the deal?

Todd, I'm going to let you take the first cut.

Yeah, see, you've got AI engines behind all of it, but at the end of the day, the generative AI is there going, hey, listen to what I'm saying and go create something for me. They don't necessarily need to do anything. We've all seen the fancy pictures. We've seen the story or Hey, I put some ideas on a piece of paper, take my nonsense and turn it into something that somebody can read and just keep going with it. That's, that's what the generative AI piece is. The agentic piece of it is, okay, I want to go do something. I want you to take action. I might give you my credit card. I might give you an account information. I want you to have a list of different agents that do very narrowly defined scopes. For example, one agent's, its job is to go scrape a web page. And figure out what you can read from it. Another agent is going, hey, go take the summary from that web page and decide what the meaning is for that whole thing, and then go post it to a website. So in some ways, it's very similar to the automation engines or the workflow engines of the past. It's just instead of being hard-defined rules of if-then, then this, you're putting AI in there to go interpret a little bit and try to reduce the number of if-then steps I have to write from 100 or 2, I've seen ones that was 3,000 different if-then steps. That's a lot of logic that you have to decide. And maybe let me put 50 if-then steps and then use AI to put the rest of it in there. I think those are the two differences there. We have the, the agentic piece versus the generative piece.

Got it. Great answer. Are, is agentic being used in the same kind of context as autonomous is? Because we're hearing, we're hearing multiple phrases here.

I think they're really close to each other. Autonomous is let it run, let it decide fully. But the real answer is, I think when I look under the covers, in most cases for most individuals, most organizations, it's really agentic AI. And you've coupled that together into an autonomous engine. You're still putting rules behind the scenes. And if you go look at it, it really is this workflow of I want you to go, you incorporate data from this list of sources. I want you to go summarize it and put it together. I want you to go further research over here, and then I want you to go take a series of actions, and then I want you to give me that output a different way. That, that in a lot of ways, that's what a lot of the autonomous AI is.

Okay.

And Todd, you and I were chatting yesterday, maybe just as a cyber-specific example, about one of the first areas— I mean, AI, agentic AI, as being tried and used successfully and not quite as successfully in various places of cybersecurity operations and dealing with the minutia of alerts and patching. Obviously it's been ripe for opportunity for a while. Even some of the SIEM providers have been using AI within their solution for years. So this isn't anything new, but just to bring up our conversation from yesterday, you had a really good point on like, brought one to your question, autonomous. Autonomous would be see an alert, follow a playbook, patch or resolve the issue, notify somebody, close the ticket. It's fully autonomous. And Todd, I think you had some tales of both sides of kind of how that might be possible and how it might be not quite yet possible yet. What are your thoughts there?

Yeah, well, we'll use the action of I see a malicious device on my network, I'm going to go lock it out, or a malicious device that I think it's happened. Traditionally, you may automate pieces of that, or there's an analyst that's in that step that says, hey, I see something that looks malicious. I'm going to go ahead and flip that switch to say block that device or isolate that device, whatever it might be. Now you have AI, and in some cases you've probably created rules to say go ahead and automatically lock that device out. But you put a lot of criteria in there of, hey, here's scenarios I don't want you to do this. For example, using data exfil as an example, let's just pick a CEO. They're out there making deals. They're talking about sensitive content. They're sharing information with other leaders behind the scenes. AI might look at that and go, danger. Yeah, danger. This is a problem. Let's go ahead and lock this device. So behind the scenes, you've probably excluded your CEO from that scenario, or you've at least sent an alert to a different person before you take action. You need to call John and say John needs to make that final decision. So those same conversations within AI, AI doesn't have that context. So we either need to provide it to it and give it the— here's the list of things you're not allowed to touch. You're not allowed to turn that safety system off that controls the fire control system. That's just an obvious thing. But it doesn't know that until you've given it that information where it understands that. And so there's this balance where security teams are going to have to figure out how much trust do I give AI? And how instead of writing all of the scenarios that you need to think through, what are the key scenarios that you need to provide that context to say, hey, if you see this fork in the road, I really don't want you to take the one that disables the CEO while he's in the middle of giving a presentation. That doesn't help anybody.

Love that as a base rule. Like, let's, let's not do that. Team, all agents, let's just establish that we're not going to screw our CEO over.

These don't help your budget.

Totally. Yeah.

So Todd, when you get into the quagmire of, you know, human notification, does that really get us any farther if a human has to verify yes/no? You might cut down steps, but if you're waking them up in the middle of the night or a ticket's created to do that verification, are we really getting any closer to real-time or 24/7 continuous to be that fully autonomous? Are we hamstringing it because we're not yet comfortable letting it have full god power.

It's, it's a two steps forward and one step back type of scenario. I'll give you an example. Writing these queries can get fairly complex. I remember one, it was 3,000 lines of query code just to go and put together a scenario of should I block this device? Where is it out on the network? Who does it own? To provide all that contextual information. That was a lot of code to write and maintain. And some of the early AI, you know, natural language search engines came online. I'm like, I don't, I don't want to update this thing. Let's see how close that they can get me. Yeah, I sat down with about 5 or 6 sentences that I put in there and it got really close to this 3,000 lines of code that I had written. And yeah, I had a couple tweaks that I had to make, but that is, that is a big step forward, right? So at the end of the day, I think that's where you start to use some of these tools to say, help me find and understand all the information that they have out there. And also I think the big next step is incorporating new datasets. So instead of— I mean, we've seen a lot of security organizations like it's really hard to bring in new datasets because it takes us time to understand that. It takes us time to figure out what to do with that. And you can use AI to leapfrog or to jumpstart that process. Now, the step back is, yep, I need a person to verify that before I have enough trust in the system that I'm going to let it run on its own. But I think everybody will get there.

So going back a bit here, I think what immediately makes people afraid is the idea that they can put their credit card information into an AI agent that they don't really know what it's doing, but they really want it to help them. And then everyone in cyber is like, maybe don't do that. Give us some context and some, some, some sort of like broad scopes here on what are, what are our actual concerns? What's the concern for everyone who's functioning in cyber right now? Very broadly speaking, 'cause that's a lot of people doing a lot of different things. And what's the concern for someone who's maybe not in cyber? Is there anything that's like lurking that someone needs to be aware of?

Yeah. Well, I'll challenge that we've put our credit card in a lot of things already. So true. Like we just do and we put it in those wallets and we store 'em there. And I would challenge that those wallets are probably more secure than some of the AI tools that do this automation today. They will get there, but they'll get there. So that's a fair question. You got to be cautious. But we also put our credit card in places that we don't trust because it's just easier. That's, that's just the deal, right? Yeah, right. Now, you do want to have controls on the other side of going, hey, where do I want to allow AI to go and do these things? And do I want it to let me know? A lot of these AI automation stacks allow you to go in and say, well, is there a filter? Is there a financial limit? And those are actually pretty easy to put in place. So that's, that's not the one that I'm worried about. The one that I struggle about is teams where they're providing administrative access to those AI tools to do things. And some of these workflow tools are new, they're young. Which means their level of security controls to get into these tools is still growing. It's, it's new, and they're going to make mistakes along the way. And at the same time, we have naturally given these platforms some pretty broad access into our environments to go look things up, in some cases modify things. I think that's the question to have those entry points for doing the administrative work. What do we know that has access to? More specifically, can we keep track of what it's doing? I think that's a bigger one that, that security teams need to have a focus on and going, how do I bring the activity tracking of where, where these AI tools are gaining access and bring it into my SIEM so I can replay what was done? To me, that's just as important.

Here's, here's my hot take on AI, and I'll speak specifically to cyber because that's an area that I've been focusing on for the better part of a decade and beyond in certain extents. Early in my career, I was in an audit role and I got to see a lot of things done wrong, 'cause an auditor's job is to find controls that either didn't exist or that did exist and then they didn't get followed. And as I was coaching people through the gaps that we were seeing, you know, I came up with this one-liner that really comforted a lot of people. And that one-liner was like, you can do pretty much anything in business that you want with the right controls and thought in place. And I think the biggest risk with AI right now is not that we should be afraid of it to Todd's point, like it's coming, it's gonna get better. I think I'll, I'll say my own line to myself and any of this is possible with the right controls and process in place and forethought. And what I think maybe the failure mode right now is like, the excitement and that initial curve of this is going to be revolutionizing. We got to get— we got to be first to the competition because our competitors will beat us. There's a lot of over-assumptions of readiness, not only the tools but the process itself. Like what, what are we automating? Do we want that to be done by agents? Is it adding any value? Maybe, maybe that's the first question. Like before we start automating, is the process just a flaming turd and we're just trying to make that go faster with less people? Or do we need to say, okay, let's check the boxes and then use the good people and SMEs that are doing lower level work than they should be already to do the stuff we never had time to do in the beginning, you know, at all. So I think I'll, I'll go right to an example right now that I think there's a lot of potential disruption and misassumption and, and that's in the space of third party risk and GRC. So what we see across a lot of our, clients and market is a lot of just really antiquated processes that are compliance-oriented or checkbox, and they're doing them, but no one really likes doing them. Enter AI, enter sexy solution, and it promises to make all that continuous or autonomous and, you know, minimize as much of the human interaction as possible. Wrong starting point. Like, yes, maybe there's some compliance things that have to exist and they have to be done because Regulation X says you have to do something. I would argue maybe you could make that more effective or efficient or business value add. But I think let's assume there's some residual checkbox that needs to be there. Okay. Yeah, sure. AI the heck out of it. Let's bring in the agents, but let's start as part of that design. Let's figure out that process and what that needs to look like going forward. So we're not just automating yesteryear's process. And then let's think, spend some strategic thought, third-party risk management. What's all the— we never had time to do like actual interaction with the third parties and combine tabletops for your top suppliers on what do we do if one of us goes down. That's where the market should be shifting to. But I don't think a lot of organizations are having those conversations. They're just looking for the easy button to do more, to do the same with less. And ultimately we're not making anything better. We're just reducing, potentially reducing humans and moving the chairs around.

Well, I, I agree with that. If you're gonna spend the time to learn a new tool, yeah, to learn a new technology, to figure it out, just to redo the broken process that you did before, that's, that's really not moving the ball. That's just spending time doing the same thing a different way.

Yeah.

And I think that's the first question. Go back to what you're trying to accomplish and you're like, am I doing it the wrong way? Should I throw away some of the process I've done and start new? So to fix it, I wholeheartedly agree on that anyway, what that's worth.

So Todd, maybe a question for me. You've, you've been into agentic AI for a while before it got the buzz that it has now. So tell me some of the good, bad, and ugly of the experiences you've had. Like give us some specifics of where you're seeing like real promising results and maybe the areas that maybe, eh, maybe not quite so right, or the, like we've talked about earlier, that there's still, you know, far too many human interactions that are being forced than what you'd like.

Yeah, I still, I love AI for here is a lot of information, please distill it down and answer the questions for me. That is a fantastic use case. I'm actually, Aaron, I'm curious your experience. Let me kind of pivot that around. I know you've done some experimentation in there. I'm curious how much, how much of inventing of information have you seen AI kind of give back to you when you're like, I gave you a bunch of information and what you came back with, something wasn't so truthful. So I'm curious your experience in that and how are you filtering through that?

Yeah, I think the bottom line, at least right now, is that it, it takes an expert or somebody that has some subject matter expertise to be able to see and spot that. Because it all looks pretty polished, right? And Todd, your question is really more about the generative AI side, right? Like feeding challenges or problems or research into it. But, you know, some of the things that I said, oh, that's not how I'd say it, or that's not actually right. I can see where it took this, but this is not how I think. That's where a human using the results and shortcutting some of their manual action or research But then like making it their own. Like I don't like thinking from a content marketing standpoint. Brahman, you've been in marketing and messaging and social for a while. No one wants to read a bunch of garbage created by ChatGPT that's right out of the box based upon some topical things. It starts to all look the same. It all starts with the same opener. It has the same outro. The form is the same. You know it when you see it. So I think really I would say to directly answer your question, Yeah, there's still a fair amount of information that you have to change and then validate. But I overall, like when I'm using it for kind of helping to spur some generation and a thought I have, I will get a few nuggets and I will get some specific copy that I might be able to use. But in almost zero circumstances would I be taking anything as an output and saying, here's my thoughts, because it, You will get called on that.

Yeah. I, I have loved it for making, you know, some of these advanced queries or asking questions of data or searching through databases that, that has been fantastic because while I've done that for years, it does take time to refine those and make them work right and check your syntax. That stuff is just, AI is making that light years faster. Like instead of spending an hour or an afternoon trying to get this one query that did things just perfect, it's 5, 10, 15 minutes and you're done. That, that's, that's beautiful. I did put together a project plan, a part list, and a, a cost structure and a whole, whole solution, but it was me just putting notes on paper really quickly. It was full of spelling errors. I'm like, AI, clean this up. And, and while it did clean it up, it did a terrible job replaying. I'm like, okay, I now have more work to fix this. So maybe I'm not going to do that next time. And again, it'll get there. I mean, every time I come in every quarter, the stuff is advancing so much. But I am I am energized by how they're— some of the traditional business process automation tools have layered AI on top of that to really start to empower, empower that work. So instead of spending your time in the nuances, did I get the syntax exactly perfectly? Did I integrate the— because integrating data is very hard. It is very hard. That's where a lot of your projects will start to run into problems, and it can make those go a lot faster. That piece is huge.

Looking more broadly out into the cyber domain, where else are you excited about or seeing actual traction in automating some of these things with agents?

Yeah, that's a good question. I still struggle with the content creation piece of it because we all know it's AI typically, and there's a lot of stuff that gets real and, but it takes a lot of human crafting to make that real and understand the content. So the storytelling piece, things like that, I think still, I still like the human authors. How's that? So let's go with that. The, the, The data side of the story, AI data cleanup, I'm very bullish around that. I mean, I'd love to see how far they'll take that into Excel. How many hours, decades of our lives have been spent in Excel? Please, please make that less. I would love it. Hey, give me a pivot table that's right the first time would be even more fantastic. Solve that, my life just gets better. I think that's a big one. And where I'm a little bit, I'm hopeful for that these integration platforms will start to use AI to generate the integration so that way programmers don't have to do all the work behind the scenes. Not that I want to get rid of good programmers, but I'd love to see if AI can start tackling ways. And they're not there yet, but I'd love to see if they can going, I know how to build interfaces that didn't exist into these different tools because what that really allows you to do is speed up going, hey, I can put something in place that can really connect my business from end to end. I think we've all been in positions where, hey, I'm partially integrated, but there's this one thing over here and it doesn't work because the data doesn't copy right, or there's just this step that gets missed. I would love to see that come to the table.

Yeah, well, Todd, I just personally, I start from a developer standpoint. I started my career as a developer for the first 3, 3 and a half years. Love the creation. I'm a builder. Most frustrated moments, hours at a time in debugging or some of the more knit and nat. So when you think about like where, where 20 years ago, where would my former self have wanted to leverage AI? And it would've been like, hey, help me find my mistake quicker so I can innovate the next algorithm or the next step. And obviously let's not encourage people to put their code into public ChatGPT, but with, proven solutions and trusted solutions that you can do that, that could be a game changer. That keep the move the ball forward, being stuck and spinning your wheels.

It was that space before the semicolon that you couldn't see on your screen that took you a day and a half to realize.

That wasn't me.

That wasn't me. Let's talk for a hot second about accountability, because I think that we've actually been great about talking about some positive stuff. When things go wrong, when Agentic AI has done things that you didn't ask it to do, when it's done its own thing, because, right, that's the idea, is like it's scraped in different ways that you didn't expect and you asked for information. Who is at fault? Who can we blame? That's what people want to know, right?

Thank the AI. Yeah, that's right, that's right, that's right.

Break it.

I— this, this gets back to a core piece of an organization should have an AI policy. And at the very top of that policy is, are you in a place and are you going to accept the outcome of AI to a customer, to a stakeholder, or is the answer— and be accountable for that, or is there need to be a human in the loop? Because I think that's a huge decision that you need to make organizationally. I don't think it really works. We're going, I'm also not going to be accountable and I'm also going to remove the humans from the loop. Those two things don't mix. That's just, that's just an excuse from the very beginning.

Yeah. And policy, and I'd chime in with governance, right? Like, no one likes the big word governance, but governance is really just a process of how decisions are made and how people communicate and how risk is managed. So I think, you know, a framework, a governance process, a, you know, review cycle with the right architects, all of those components are helpful. When I mentioned you can do anything with the right controls, Don't wing it and hope for the right controls. Like, have some process, have some expertise to build that so that you're not wasting your time or having a chutes and ladders where you fall down in the pit and you spend a year climbing back up and then you're back to where you started versus kind of a steady stream of progress.

Progress. Interesting. And progress seems like it doesn't start with AI. It sounds like it starts with humans rethinking what we actually want to achieve.

Yeah, let's, let's We can leverage the robots and AI to help us envision the future through generative. But let's not take today's mess and have agents do it cheaper and faster because it's only— we're going to have problems and it's not going to work perfectly. Use the time to innovate and add more business value. Otherwise, we're just, we're just a cost savings engine.

I have seen some projects where they start with the data needs to be right. Before we get into AI. And that, what that's really code for is I want to see the process right-sized before, before I start this project. And that does make sense. I would say there's one kind of caveat to that. If it's, if it's truly a data issue, sometimes you can use AI to go fix the data and clean stuff up. There's some simple examples there, like address creation. Companies spend dozens and hundreds of millions of dollars just to fix their address table that they mail things out to or communicate to customers and make sure that's right. A lot of money is spent on that. There's services and third-party tools that do that, and it costs a decent chunk of money. AI is being used to accelerate that. So I think that's an overly simplistic example, but I think if you're starting with, to your point, Aaron, if you're starting with a broken process, if you're starting with broken data, you've set yourself up for potential failure there, or you've at least tied one arm behind your back.