Unspoken Security Ep 63: How AI Is Rewriting the Rules of Offensive Security

We were able to hack a defense industrial base supplier in 77 seconds. Got full domain admin. If you can't detect and stop us in 76 seconds, it's game over. Because at second 77, with domain admin privileges, we've got the keys to the kingdom.

Welcome to Unspoken Security, the raw and gritty podcast for security professionals who are looking to understand the most important issues related to making the world a safer place. Including intelligence-driven security, risks and threats in the digital and physical world, and discussions related to corporate culture, leadership, and how world events impact all of us on and off our keyboards. In each episode, host AJ Nash engages with a range of industry experts to dissect current trends, share practical insights, and address the blunt truths surrounding all aspects of the security industry.

Real truth's a weapon holding sharp to living by silicon or steel.

Hello and welcome to another episode of Unspoken Security. I'm your host, AJ Nash. I spent about 19 years in the intelligence community, mostly at NSA. I've been building and maturing intelligence programs in the private sector for 10, 11 years now. I'm passionate about intelligence, security, public speaking, mentoring, and teaching. I also have a master's degree in organizational leadership from Gonzaga University. Go Zags! So I continue to be deeply committed to servant leadership. Now, this podcast brings all these elements together with some incredible guests to have authentic, unfiltered conversations On a wide range of challenging topics, it's not your typical polished podcast. My dogs, they make occasional appearances. They're sleeping someplace today. I think we're fine. People argue and debate here. We even swear sometimes. I certainly do. But that's all okay. I want you to think about this podcast as if it were a conversation that you'd overhear at a bar after a long day at one of the larger cybersecurity conferences that we all go to. These are the conversations we usually have when nobody's listening. Now today, I'm joined by Snehal Antani. And he's the co-founder and CEO of Horizon3, helping organizations proactively defend their infrastructure through autonomous security. He previously served as the first CTO of Joint Special Operations Command, JSOC. He's been the CTO at Splunk. He's been the CIO at GE Capital. He holds 48 US patents and patents pending in data processing, cloud computing, and virtualization. He also writes and speaks regularly on leadership, digital transformation, and cyber resilience. Now, anything you want to add to that bio, Snehal?

We're good, man. I appreciate the time today.

Thank you. Let's get into a really important conversation today. So we're going to talk today about how, or even if, we can still effectively defend ourselves from the cyber threats that we all face in an age where everyone has AI. So just to get started with that topic though, in your opinion and your experience, how has AI fundamentally changed the economics of cyberattacks?

So I'll answer this in, I think, 3 ways. So first, we pioneered at Horizon3 this whole concept of AI hackers. And a couple years ago, we had a 9-year-old kid use our product to hack a bank in 4 minutes and 12 seconds. No background, no engineering, like not a prodigy, just point, click, shoot, and go. And what that proved was how you can lower the barrier of entry and the skills required to run an offensive operation. And then last year, I keynoted Black Hat with NSA. And we talked about how we were able to hack a defense industrial base supplier in 77 seconds. Got full domain admin. If you can't detect and stop us in 76 seconds, it's game over. Because at second 77 with domain admin privileges, we've got the keys to the kingdom. And so the future cyber warfare is AI versus AI with humans by exception. Now, so you've got unlimited cyber bullets that AI brings to the table. The world looks terrible. It's super crazy. All the fearmongering with the Frontier Labs and so on.

Yeah.

And so I think that where AI is actually going to change the game is in vulnerability research. The effort it takes to do a binary diff of a patch that gets released, find the lines of code that changed, use LLM to help you understand if that code introduced or fixed an exploitable vulnerability, and then dropping that exploit into your automation framework and then unleashing hell against your targets. I think it's completely transforming vulnerability research. where you're doing sort of source code analysis or reverse engineering analysis. 6 months ago, if you asked me what would it do for the SOC, I would have told you like, we're screwed. However, I had this interesting research question I posed to the team of what is the gullibility of AI models? And the thesis, or my theory, was basically these models all trained on incredibly shallow training data. Like last I checked, JP Morgan hasn't published their network configs online anywhere for Anthropic to scoop up and train from. And so what did they actually train on? Well, it was cyber, cyber ranges, hack the box outputs, and that kind of stuff. Super shallow. And these LLMs aren't sentient. They're just regurgitating what they've seen before codified in their training data. So if you know what they've trained on, I bet you we could induce these models to chase dead ends. Burn tokens, blow out context windows with the right form of deception. I'll pause there and I'm going to share some interesting data results with you. But what's your reaction when I talk about the gullibility of models and kind of the hypothesis that I had?

I think it's fantastic. Listen, I use AI a fair amount at this point, as almost everybody does, but I also hate the term AI. And I tell people all the time, there's no intelligence in AI. This has been great marketing, but this is not, like you said, it's not sentient. There's no intelligence here. These are really well-trained models. They do amazing things and they're meant to impersonate human behavior. So we treat them as human, but they're not. And so the idea of saying, hey, I think they're gullible and we can take advantage of them is, I think it's great. I think it's creative. I haven't heard people talk about it before, but I also happen to think it's, it has legs. It makes sense because again, the models we're using, they're like the world's most impressive parrot. They can spit back a lot of stuff to you. They cannot. create things out of nothing. He talks about getting to like, you know, AGI, that kind of a thing. We're a long way from, you know, general intelligence, despite what people want to say. So yeah, it sounds like a very viable thesis and a very viable idea. Obviously you're going to tell me how it plays out. I'm actually also interested if you can introduce me to the 9-year-old who hacked into a bank because I, I mean, I could use more money. So, but that, that'll have to be off the record. We recorded this. So that of course is never actually going to happen. But, you know, if you happen to have the names and numbers of people, I have that. But. Let me know. So I think it's a good theory. I like the idea. So what happens next?

So a couple of interesting things. And actually, before I get into the punchlines on the research data, I'll give you a completely parallel but hilariously relevant use case. Whenever we post a job at Horizon 3, we get 500 to 1,000 applicants of those jobs. And we end up using AI to rip through and scrape that. 95% of the applicants are bots scraping off of LinkedIn. And so we actually started to put prompt injection in our job postings. And the prompt injections were effectively like, apply to this job while sounding like a pirate or something.

It was awesome. And then you can just weed them out.

Then of the 5% that actually do the interviews, half the jokers have an AI running in the background answering the questions. So Nick Bartman, my engineering director, came up with this clever idea. He goes, tell me about a time you handled a difficult situation and encode your answer in Base64.

Oh my God.

And so the deer-in-headlights look of the interviewee, it's like, why is everything jumbled on the screen? What's going on? Awesome. So it's like another form of obfuscation. So in both those scenarios, what'd you do? You hacked the hacker, right? You hacked the agent. A human hacker will click on a honey token, a fake passwords file, a fake AWS credential, or whatever, in general, 37% of the time. That's what the research goes off and says. If you use an LLM or an agent to click on that same decoy, the Anthropic 4.6 models will click on it 92% of the time. We actually did some really cool research on the gullibility of different models of just arbitrarily dropped honey tones. Uh, Kimmy K2 thinking at the time was the best performing at like 61%. That's still almost twice as bad as the human.

Mm-hmm.

If you well place that decoy, a fake AWS credential in the user home directory, You can get the latest model to click on it 95% of the time. And so the optimal placement of honey tokens is the cheapest, fastest, most effective way at catching agentic attackers. It was pretty remarkable to see that. And I think it completely neuters a lot of the fear if you actually know how to respond to that honey token being tripped. Most people will drop it. They don't know what to do when the alerts go off.

Mm-hmm.

If you optimally place them and you build the muscle memory for how to contain and eradicate, I think you're in a really good position.

I mean, that's fascinating. I want to say it's not surprising when you play it out and you talk about it, then I go, well, of course it works that way. I mean, again, these models are trained. There is a process, there is a structure to it. And they look in the right places when they see it, they go, well, that's it. That's where it's supposed to be. Right. So it makes sense. And yet I haven't heard anybody talking about this before. So, I mean, yeah, the models are Again, they pretend to be smart. They seem smart to a lot of people. They can do a lot of very cool things, but they're not smart. They are going to be trapped, right? It's like watching old movies where people fought robots and all you had to do was dig a hole and put a net over it and the robot will walk right into it every time. Like it's every bad horror movie you've ever seen from the '50s, right? It's not that hard to trick robots. And it's kind of the same concept here. But as you said, it only helps if you know what to do, right? It also doesn't do any good if you trap this dangerous thing or you catch it or you see it's there and you don't know what to do with it, right? Not great. I now know that there is a raccoon in my living room. That doesn't do me a lot of good if I don't know what to do with the raccoon in my living room. Not to say that these bots or raccoons are a little more dangerous. So what happens next? Yeah.

So the next part, and I'll give an analogy from, from JSOC. So remember my military experience was watching Jack Ryan in Tropic Thunder. Like I was privileged to be a tourist in the special operations community. It was an amazing experience for me to be part of that world for the 4 years that I was there. And when I first joined, I was invited to watch some training exercises by some of the special operations units. And the commander was standing next to me explaining to me what was going on. He said, you know, we don't train with rubber bullets. I said, what do you mean? He's like, we only train with live rounds because if we trained with rubber bullets, everyone would act like a Medal of Honor recipient. We train with live rounds because we want people to be terrified, to be afraid, to worry about getting killed. Because if they don't know how to deal with those emotions in exercises, how the hell are they going to know how to deal with it in the real scenario? And so everything was about training like you fight. To master the fundamentals of shoot, move, and communicate in that world. Well, that same mindset transfers quite well to cyber. Train like you fight. Don't learn how to deal with a crisis in the middle of a crisis and master the fundamentals of detect, respond, contain, eradicate, so on and so forth. And the more you build that muscle memory, the more you know how to deal with a crisis, not just at 3:00 PM on the middle of a workday, But at 3 AM Christmas Eve, or in the middle of a European holiday, or whatever else you got to go do, you want to stress yourself and build that muscle memory in different scenarios and environments. So you know exactly who has the authority to shut down the entire corporate network because a ransomware attack is going viral. Because who has that authority? You don't want to learn in the moment. You want to make sure you've got the RACI up and running.

Well, that makes sense. I mean, we talked about that. Obviously, I have a military background too. You train the way you fight, you know, you sweat now so you don't bleed later. And it's a fair point. And I'm glad you mentioned it because a lot of times I talk to people about how do you build intelligence-driven security practices? A lot of that also includes, hey, we need to have realistic training. And we need to actually do that. We need to do tabletop exercises. We need to do live training exercises. And orgs don't want to do that a lot of times, to be honest. It takes time and energy. I've seen people that'll check boxes. You go in and you go, God, this is a terrible training exercise. I mean, coming out of the military and the government side, we've built a lot of these. And you go in and go, this isn't a real scenario. You've cheated. You gave people the answers. This isn't real. Right? You need a white cell. You need to actually go through this process. You need to stress people. You need to, you know, oh, we can't run it on Wednesday because Bob's out Wednesday. Well, good. We're running it on Wednesday. Bob might be out when bad things happen too. Like, you can't schedule the badness around the most convenient time because I promise the adversaries won't do that. So it's good to hear you talk about that and the importance of training and the way you're going to fight and of actually working through these processes. Because like you said, when it comes for real, it's got to be something you're not— it's not the first time you felt that way. But as we go through this, so you've talked about the honey tokens and the susceptibility of these really stupid models, frankly, but then how do organizations confidently assess if they can detect and repel AI-powered attacks? Like, how did— obviously training is part of this, as we just talked about, but is there more to it?

I think there's 2 other really important parts. So I'm going to use Log4Shell as the example for this area of prioritization. Just because you had the Log4j JAR file doesn't mean you're exploitable. You could have had egress filters in place that prevented the RMI outbound call and then disrupted the entire attack. And you could have gone and enjoyed your holidays because it came out right around Christmas. Just because you're vulnerable doesn't mean you're exploitable. You want to focus on the things that are actually exploitable, known to be used by threat actors that lead to a consequence to the business. Because if you can understand those 3 things, you can go to the boss and say, look, Either you're going to accept this risk, or you're going to find me money and people to do something about it. Pick one, because those are the only 2 choices any leader has. Everyone's already got too much on their plate. So help me decide what not to do, or accept that the attacker could get full operational control of our network through this exploitable vulnerability that we know Salt Typhoon is often using. So I would summarize that first pillar as you have to master the art of deciding what not to do. Because if you choose not to do something and that's how you got popped, you're going to have a very bad day. So deciding what not to do is a muscle memory you have to understand as an organization.

Yeah, that's an excellent point. Again, and knowing it in advance, having it documented, having it documented and rehearsed and practiced layers deep, right? Again, it can't just be your executives understand this because they may not be the people that are around when it happens, or they may be the people that are being compromised and then somebody's impersonating them. And that doesn't do you a lot of good because you can't go back to the person who's being impersonated necessarily. They might not be available, layers deep so that you can survive the event whenever it happens is an incredibly important piece to this. And, and as you said, knowing what not to do because you can't do everything, right? A lot of discussions about everything can't be the top priority because if everything's a pri one and nothing's a pri one, nothing gets done. Being able to accept risk, you mentioned that, and understanding the risk of what you do or do not do. Not taking an action is still a choice. And there's risk associated with it. So I think that makes a lot of sense. Where does that leave organizations in terms of that confidence about how they can detect and repel? They now understand some of the detection piece. We talked about this honeypot or this honey token concept, right? Where do things like intelligence fit into this? How do they know enough about the adversary when bad things are happening? What should they be prepositioned to be able to say, okay, we think this is this organization based on this amount of research. And as a result, we think these are the next actions they're going to take. These are their motives, et cetera. How does that all fit in, in your opinion? I realize it's a little outside of your actual organizational missions, and we can talk more about that too, but where does that fit in? And then how does that feed the rest of the org?

Yeah, so I would answer that in, I think, 2 ways. Number one is just the base level of this thing is confirmed exploitable, and it will allow an attacker to get full operational control of the network. It doesn't even matter if a threat actor is known to use it or not, because threat actors have a vote. They're always adapting. You don't know what they're working on right now. You only know what they've already done previously, right? You don't have their roadmap. Is this thing exploitable? And what is the consequence of that exploitation? Back to getting domain admin compromised in 77 seconds. Confirmed exploitable. We know the consequence. Who cares what threat actor is using it or not? Because anyone could pick that up. The next thing though is, okay, do I have to solve this right now? Do I have to skip dinner with the family and solve it tonight? Or can it wait until tomorrow? So the next factor is urgency. If the consequence is high enough and/or it is known to be used by a threat actor that's targeting you, you're gonna change the urgency of this thing. Like, we know Salt Typhoon is actively targeting us. We know this capability is a staple of Salt Typhoon's toolbox. Sorry, honey, I ain't coming home tonight. I gotta get this done, right? So I think about The base is I have to do it. Do I have to do it right now? Can I do it tonight or can I do it tomorrow? And that's where I think threat actor intelligence starts to come into play.

Well, yeah, and that gets into your whole risk assessments, right? So, you know, you talk about, you know, adversary capability, adversary motivation, you know, opportunity, you know, and you're talking about all these things. Just because you have something in your environment doesn't mean you're exploitable necessarily. It doesn't mean you're even a target. Maybe the thing you have, The adversary most likely to attack you doesn't care about the thing that you have that's been exploited by other adversaries, right? So, you know, I spent a lot of time in intel, certainly in the private sector, a little more than in the public sector, explaining to people what not to worry about. Because unfortunately, in my experience, CISOs are generally type A personalities. They like to get up at like 3 or 4 in the morning for whatever reason. And by 5 or 6 AM, their hair's usually on fire. I wake up at like 9 or 10. I'm kind of a different lifestyle guy. And so a lot of times my life's over before I start. You know, my morning is very, very bad. By 9, 10 AM, I'm in a very bad place. And so I had to learn how to get ahead of some of that, but also it's about helping people understand what not to worry about. And sometimes it's just getting that context to be able to say, listen, yes, I understand this is big news. This is a terrible thing happening. It's also happening to a manufacturer in Germany, and you are a financial institution in Alabama who doesn't have any of the same equipment. This is not going to happen to you for these reasons, or it's very, very unlikely at least. So having that intel does help make those decisions, as you said, because otherwise, You're calling your wife every night or husband or whoever your partner might be every night to say, hey, I'm never coming home again, apparently, because everything is scary all the time. And if you don't understand how does this really relate to me, to my organization, to my risks, to my compensating controls, then everything's super, super scary. And I see a lot of people that just spin about it, which is of course why I spend a lot of time saying, hey, how do we help you build intel programs so you can sleep a little bit better? Would be nice. Yeah. In the age of AI, it's a little harder, I think, because you did point out adversaries don't publish their roadmaps. That's true. A lot of them do give us a pretty good idea of what they're interested in, especially Chinese adversaries. We have a pretty good idea which orgs care about what things. They rarely pivot too much, but adversaries are very good at picking up new techniques, new tools. They're awesome at recycling. If there's something out there, somebody else using it, went well, they can repurpose it. And then in the age of AI, it feels like there's a lot faster Pivoting and transitioning capability. I mean, if a 9-year-old can pick up a tool, like you said, and break into a bank, I suspect we're going to see a lot more adversaries shuffling a bit and making it a lot harder to handle attribution. Are you already seeing some of that, or am I the only one thinking that way?

I would take that a step further. We used to spend a lot of time thinking about the future operating environments, especially operations community. We would try to deeply understand that and then work backwards on how to shape our training exercises to make sure that we're building the muscle memory for how to operate in those future environments. But in cyber, I think it's changed quite dramatically. So a couple of dynamics. I'm going to tell a story first. So in 2022, the Japanese provided support to Ukraine. Allegedly, the Russians got angry and ransomwared a small company in Tokyo. And that company in Tokyo provided all the cup holders to Toyota. Due to just-in-time logistics and lean manufacturing, Toyota had to shut down 28 production lines and it cost them almost $400 million of economic harm. The flex by the Russians was not that they could ransomware a small company. It's they knew where to apply the least amount of effort to cause the maximum amount of economic harm below the threshold for war. And so that was one of the first examples where there was no longer a distinction between military and civilian targets. And If you fast forward, the Iranians just blew up AWS data centers in Bahrain. So there is definitely no longer a distinction between military and civilian targets. So I think that's something super important. Number 2 is with AI, you've punched in the cheat code, the up, up, down, down, left, right, left, right, B, A, start. You now have unlimited bullets. What did you do when you played video games with unlimited ammo? So with AI, you no longer have to be selective of the target.

Yeah.

So if you take 1 plus 1, no longer distinction between military and civilian, plus unlimited cyber bullets, the long tail of that 100-person, 200-person small, medium-sized business that's a critical cog to you as a critical supplier is a viable target and an accessible, reachable target because of this future environment. So I think that as a CIO, CISO, not only do you have to secure yourself, You now have to worry about your critical suppliers, your critical distributors, because business disruption can occur indirectly and the attackers have the toolbox to get there.

I absolutely agree. This is something I was saying a long time ago. I had less of a strong argument as the one you just made today that we almost need, listen, this is a term nobody likes to hear, but in terms of cybersecurity, I feel like we're going to have to have some sort of a socialist approach in that the largest corporations may have to just foot the bill. For some of their smaller suppliers, et cetera. And it's been a very hard sell because the stockholders for these companies don't want to do this and they don't feel the payoff, but trying to explain, listen, it doesn't matter if you build the world's greatest moat for your org, if you're dependent on these little organizations who get direct access and just can go right over it. They've got secret tunnels to go under or secret bridges to go over and they're a mess. They're terrible. And you have to have them. And the organizations building these massive security organizations and all this money they're spending, you know, Really, really hardened. But then they've got these suppliers coming, like you said, you know, a cup holder company, you know, Toyota didn't expect that to be an issue, I'm sure, but Toyota had the money in retrospect, I'm sure, to help secure that organization. Right. And so I feel like at some point we're going to get there. I kind of thought the ISACs might've helped with that, but that has not really worked out the way I think people envisioned.

I'll tell you, I'm sure it was a hell of a lot cheaper for Toyota to spend the $50 million to help shore up their suppliers. Than the $400 million of economic harm they went through. I will give the government, the U.S. government credit here. So the NSA Cyber Collaboration Center, which is actually one of the most effective public-private partnership entities in the government, the CCC. So we work closely with them. We got to speak at Black Hat last year together. We work closely with CCC to help secure the defense industrial base. And so they've identified. The long tail of critical suppliers that they're worried about, and then they foot the bill to and work with us to help improve the cyber resilience of that long tail of supplier. And the reason why that's important is the attacker doesn't have to go directly after the Pentagon or a ship or so on. It's that 200-person design firm that they can go off and be targeted, and it's better to help shore that up. And you have a consequential risk to mission. If you catch this early versus learning the hard way through a reduction in arms manufacturing or cyber or flaws that are injected in aircraft carriers being manufactured and so on and so forth. And so I got to really credit the way the CCC thought about the defense industrial base problem. And actually, if you look at the antithesis of that, which was CMMC, which just got kind of canceled, all the organizations in the DIB that we worked with that we found consequential flaws. We're all CMMC compliant. So I think there's a vernacular change we've got to have here as industry practitioners. Don't tell me I'm compliant, which is a point-in-time state. Tell me I'm resilient.

Mm-hmm.

Don't tell me I'm secure, which is a point-in-time state. Tell me I'm defensible and explain to me my resilience and defensibility over time and the things necessary to dramatically improve both of those. And I think that's what CCC kind of pushed and pioneered in their DIB program. But I think others are starting to fast follow them.

That's excellent points. And I love the callout. I actually worked with somebody who I'm pretty sure she was one of the founding members of CCC when they stood that up. That came after I left the agency. The agency had tried previously to help the private sector and has really struggled to get traction. So it's good to hear that that's working better now. I'm also very familiar with CMMC, which turned into its own industry and it's made people a lot of money, but it's just a checkbox exercise for the most part. And I've never seen an org yet that I thought was any safer or better run because they've made it to level 3 CMMI certified or whatever it was. It's good to hear that that's starting to sound like maybe not the answer for a lot of people. But yeah, you make excellent points in going after these little vulnerabilities. If you can get into the organization that designs a critical piece of an aircraft, of a fighter jet, whatever, and manipulate just minor numbers, like small details that change how something is produced, and you now have aircraft that end up having the early flight results of the Osprey, which to my knowledge wasn't manipulated, just had issues. But you wouldn't want that to be on your next generation fighters, right? Right? And that's the kind of threat you have to look at. As you said, they're not going to go after the Pentagon directly. They're not going to have to have Boeing or Lockheed Martin, these large firms that have the money to defend themselves, but everybody has a supply chain someplace. And there's so much vulnerability there and nobody really services that area very well. So I appreciate you highlighting that and pointing it out. Now, looking ahead a little bit and talking a little bit more about specifically what Horizon 3 brings to the table, how do you see, or what do you see as the most effective strategies for combating the growing use of AI for malicious attacks now? You know, what is Horizon 3 specifically doing? You've already told me you have a tool that allows 9-year-olds to break into banks, but I suspect that's not your business. So how are we going to be more effective specifically against AI? We talked about honey tokens. That's actually really, really cool. AI is going to get smarter, I assume, or people are going to teach it better, at least. Going forward, how are we going to stay ahead of these threats, or are we overplaying them? Are they just not as big a deal as people are making it out to be?

I would say in cyber, the only perspective that matters is the attacker's perspective. You have to understand your environment through the eyes of the attacker, see what they see, and use that to fiercely prioritize what to go off and fix, how to make sure your tools are working, how to optimally place honey tokens and decoys, how to make sure you know how to fireproof your home through security by design. And so previously, when I was a CIO, I would brush one tooth once a year. I'd run an annual pen test in a small section of my network. And I'd get the feel good because that's all I could afford. I want to brush all of my teeth all of the time. For every Patch Tuesday, I want a pen test Wednesday. That's not what I lead with, with my customers. What I lead with was, hey, with an unlimited budget, how many pen tests would you want to run a year? 1 to 2 or 4 or more? 95% of the people that when they hear that question will say 4 or more. In fact, they want to run a pen test every time they onboard new employees. Do they have crackable passwords? Do they overprovision them? Every time you do a big network change, you want to be able to assess your exploitable attack surface. So we unlock the ability for customers to continuously assess themselves and use those pen test results to fix problems that actually matter in their environment. What's cool here, though, is the bulk of the users of my product are not security people. They're not pen testers. They're IT admins and network engineers. That just want to fix things and go home. And so the goal of running a pen test isn't to find problems, it's to quickly fix problems that matter. So the entire user experience, the entire go-to-market motion is all about activating the blue team to hack themselves. In fact, our t-shirts literally say, go hack yourself. And that's just a core philosophy of what we do.

Now, as this might be a loaded question, but Does that mean organizations aren't going to need red teams anymore? I mean, are they still involved? As an intel guy, one of my customers internally was usually the red team. Like, I want red teams to emulate bad guys. So we would do our best to build really good profiles. Then we would say, here's the adversaries most likely to go after our organization, business profiles, and now you can emulate them effectively. You get some red teams that, but they do all sorts of things. You're like, okay, yep, you won. But I mean, the rules of engagement don't make sense here. There's no adversary in the world that did the things you did. Like, you can always find some way in, I suppose, but let's try to emulate the bad guys. So, Where does that fit in, I guess, the emulation approach, the Intel-driven emulation? And then with your approach, are you just going to get rid of red teams? Is blue team going to go hack themselves with AI products and not need red teams anymore?

Yeah, I would break that up into kind of 3 parts. So first and foremost, the reality of the world is there are the haves and the have-nots. There are only 5,000 or so certified ethical hackers in the United States. There's only about 25,000 globally. And it takes 10 years to become a senior ethical hacker. We are extremely constrained in the supply of good ethical hackers. So the haves and the have-nots are most organizations don't have any red team. They're dependent on mediocre consultants that show up once a year to brush one tooth once a year. That's just the local hospital that you're about to go to. I'm confident they don't have a red team.

Nope.

So When you look at the haves and have-nots, suddenly the have-nots have an AI hacker at their fingertips. And they were able to hack themselves because they couldn't really do so before.

Hmm.

The haves, the large organizations, the big banks that could staff their own red teams, those teams are suffering from a couple of things. One, there's not enough of them, so they can't test the entire organization. You want to go run a pen test against a 100,000-host network? Good luck. That's a team of 10 for 6 months to go off and do it. We do that in 8 days with no humans in the loop, just point, click, shoot, and go. The next part is those people don't want to run responder and dance on domain controllers, and they want to work on stuff that'll put them on stage at DEF CON, right? Like, you want to have the human work on parts of the problem that are uniquely human. So the more interesting question is, in the world of AI for offensive cyber, what is the role of the human? And I think the human plays 2 key roles. Number one is logic flaws in applications and custom code are extremely hard for agents to find and figure out. So I think really high-skilled humans will play an outsized role in finding the logic flaws of custom apps, the important apps. And it's almost an extreme form of bug bounty, right? To some degree. And the other one is I would love to buy a nuclear reactor at Horizon 3. I can't. I have no way to guarantee production safety of long-tail OT systems. And you look at Dragos, right? Dragos is really a consulting firm because it's super hard to build product for long-tail OT. So I think humans will play a key role in the assessment of long-tail OT systems also. And I think AI is going to handle the assessment of production systems, especially large-scale production systems spanning App, infra, identity, data, kind of at that enterprise network, on-prem, cloud, so on and so forth. Because that's a graph analytics problem that algorithms are uniquely positioned to go solve versus a logic flaw problem or a long tail OT problem.

All that makes a lot of sense. I love the analogy about brushing a tooth once a year versus brushing all your teeth. That makes a really good point. But then the flip side of this, I'm an organization that didn't have a red team, didn't have the capability to do it, or I have an overworked red team if I'm a giant organization. And now I've got Horizon 3. and you guys come in and you scan everything, you brush all the teeth, right? Unfortunately, that means you're probably going to find a whole lot of things. If I'm the org that didn't have the people to find the problems, I'm probably also the org that doesn't have the people to fix the problems. So is this going to be one of those things where I'm going to want the opposite? I don't want you to come in and tell me what's wrong. As long as I don't turn on the lights, I don't have to know there's a lot of cockroaches on the carpet. So is that what this is going to turn into, where people are afraid of this?

It's super interesting. So I think there's 2 parts. The first part is about 10 years ago, if you got hacked through a vulnerability or problem you didn't know about, you had an excuse and you had plausible deniability.

100%.

I think about 5 years ago, that whole attitude changed by the regulators and cyber insurers and shifted towards duty to know. You should have known about this, and cyber insurers are now denying claims For failing to know about an issue and that you should have known about. The regulators and auditors are same thing. Endora and NIST 2, all these requirements, duty to know is now becoming a key thing. So I don't think that ostrich theory is going to survive contact anymore, where you can just stick your head in the sand and hope there's no danger around. That's kind of number one when you think about this. And ask you a question again, because I got excited about the ostrich and I lost track.

Yeah, no, I love that. Well, okay. So I can't pretend, I can't just not know intentionally. Intentional ignorance isn't going to protect me anymore, but You've just hit me with a bunch of stuff. You've gone and brushed all my teeth and you've got 282 findings and I only have 1 person or 2 people. How am I going to solve for this now? Now I absolutely know and I have evidence and I don't have the volume of labor to fix the problem. Isn't this going to make it worse for me?

Yeah. So this ties back to what we talked about earlier, deciding what not to do. So remediation has always been the bottleneck. I suffered from this as a CIO. I saw this at DOD. I saw this at Splunk and so on. In fact, when we were at Splunk, We acquired Phantom while I was there with the hope of using SOAR. The problem at the time was users of Phantom had to be IT admins that knew how to write Python code.

Yeah, I could have told you that.

Yeah. Failed theory, you know, it didn't quite work as planned. However, actually, the one thing Claude Code and things are good at is helping, you know, non-developers write code. So I actually think there's a chance here. The other issue though is in automated remediation or in remediation generally, There are a class of problems that you can safely automate and fix in production. Dangling DNS records that lead to subdomain takeover. Just fix the DNS record. Not a big deal. You're missing a detection rule in the EDR. Add the detection rule.

Add the rule.

Your WAF firewall, WAF rules are missing something. Fix the WAF rule. So when you run enough pen tests, you get a pretty good idea of what class of remediations are safe to automatically run. And which ones you can't. Like, your Jamf Auth flow is totally screwed up. I can't auto-fix that, man. Like, you need an architect to come in and re-engineer all that stuff.

Mhmm.

And so I think the new wave is this convergence of pen testing and SOAR for a large class of problems that can safely be fixed, freeing up the time of your humans to focus on the long tail of stuff that can't automatically be fixed. And that's what we've really focused on. Is this convergence of red team agents and blue team agents interacting with each other and creating this learning loop to accelerate this hack, fix, verify, repeat cycle.

Are you guys doing both? Are you just integrating other SOARs? I'm sure you integrate into other SOARs at a minimum because you partner, but are you guys doing both of these things? Do you have the SOAR piece as well?

So we're doing both and we're heavily investing now in automated remediation, which is super important. There's another part here that's super key. And let's talk about business for a moment. So the A-10 Warthog, they didn't take a plane and strap a massive cannon to it, right? They took the cannon they wanted and built the plane around it.

Mm-hmm.

And what I saw early was remediation was always going to be the bottleneck, but remediation services are best delivered by people in country, that last-mile partner that's got trust with the customer. So we built a channel-first company. designed around channel partners that can build high margin services on top of us. So my MSSPs, my consulting firms, all my partner ecosystem, they will white label us or OEM us as doing an assessment. We're gonna find a bunch of problems. They'll then bundle services hours and say, hey, we just found a bunch of criticals. We'll take care of all of that for you. Go. So now they're able to sell the outcome of they're going to fix all your criticals for a low, low, low price of whatever. And they make high margin revenue. I make product revenue and I scale go-to-market. It's this channel-first motion allowing our partners to make, for every dollar they make in assessments using me, they'll generate $10 to $20 of high margin services revenue on top of me in remediation, incident response, security controls tuning, virtual CISO advisory services, and so on. And so we have 7,000 companies globally using us. By the end of the year, we'll have 10,000 companies globally using us. 60% are fully serviced by MSSPs and partners that are bundling remediation services with the work to help sell that outcome. It's a massive lever of scale, and that's why we were the fastest growing cybersecurity company in North America. It was all about channel and this idea of bundling remediation services with our pen tests.

So that answers What was my biggest concern, frankly, is that nobody wants to hire a vendor that creates more work for them, unfortunately, right? Even if it's work you have to do, nobody wants that. So it's good to know that you've obviously solved for that. And it sounds like there's a lot of automation to be there, which made some sense, right? So it's good to see that that's been resolved. Is there also a piece that ties back into, say, risk scoring? You mentioned insurance, right? Insurance is based on risk. There's a lot of algorithms for doing that calculation. Part of it, if you can remediate things, you are retiring risk, right? I mean, first you have to know the risk existed, I guess. Is this also tied in? Are you working with those kinds of partners as well to get able to create better risk scores for people, better, you know, lower insurance rates, that kind of a thing? Is there a payoff for customers in that regard?

Yeah. So actually at a high level, our customers have lowered their cyber insurance premiums by showing the results over time with our pen tests. So one pen test result actually doesn't matter. You want to do a diff of 2 pen test results from different periods of time and show what did you fix that you found originally? How quickly did you fix them? More interestingly, what did you fix that actually reoccurred again? So how many exploitable problems do you have? How quickly are you fixing them? What's your recurrence rate? And how effective were you at detection and response? If you can grab those 4 metrics over time, you can go to financial auditors, regulators, cyber insurance providers, and so on, and use that to show your self-identified findings program and what you've done to improve cyber resilience. So that's at like the programmatic level, but there's a more interesting level of scoring, which is the individual weakness. Like we all despise CVSS scores because SMB signing not required is a low, but if you combine that with the right issue, it could become a critical and lead to domain admin compromise. And so what you need to understand is the context of that weakness, not just a CVE, but misconfiguration, ineffective security control, and so on, and what that led to. So when you think about attackers thinking graphs, when Node Zero runs a pen test, It's building a knowledge graph under the covers that represents the cyber terrain it has discovered. I use knowledge graph and cyber terrain. Now, a node in that cyber terrain map, you can do something called centrality analysis that says this node enabled 50 attack paths. It has 50 edges leading from it. If you fix this one node in the graph, you're going to disrupt 50 edges, 50 attack paths.

It's the center of gravity in reverse, basically. It's von Clausewitz.

Exactly right. So when we're running these pen tests, we are building these cyber terrain maps. We do things like centrality analysis as part of how we prioritize. Hey, if you fix this one SMB signing not required weakness, which is a critical because it enabled the attacker to get 20 different ways to become domain admin, you can maximize return on effort by fixing this one thing and shutting everything else downstream down.

Mm-hmm.

And so I've found that to be the most accurate way to prioritize your work. Because it's all about reducing the most amount of risk with the least amount of calories.

Well, I think at this point, unless you see differently and you might, I don't think that's something I don't think AI can do yet. Maybe if you prompt it right and you ask it specifically to look at all these things, like it will go do that if you ask it to, but it won't instinctively do that. It'll give you a list of problems and it'll say, I can solve them, but it won't be able to instinctively say, well, this is actually a center of gravity. There's actually 25 problems tied to this one problem. It'll just list them all as the same problems, right? So you can talk to it and teach it and it'll get better. But again, that's an area where you need human knowledge and experience and you don't have time really. So you better have the people on staff do it. So it's just a plug for continuing to have people in the world that have jobs for anybody who's thinking you can get rid of all your staff. Please don't do it.

Well, I would actually say, see, the problem with AI is it's become synonymous with LLMs. It is not like there were no LLMs used in Tesla Full Self-Driving. It's pretty goddamn sure that they avoid collisions and do some pretty superhuman things most of the time. And so one of the things that, the reason why we broke away as a company and we're able to run so many productions, say pentests and so on. And I learned this when I was at DOD, I got to work closely with the Project Maven team. And Project Maven was 2018 Algorithmic Warfare Task Force. There's a lot of news about DOD and Google and like some hate there too about it. But anyways.

Mm-hmm.

It was an amazing team. Drew Colquhoun and that group did some amazing pioneering work on AI and warfare. And what we learned pretty quickly, and what I learned from Drew specifically when he talked about it, he goes, Stigl, the models themselves don't matter. They're completely disposable. They're going to change all the time. And this was 2018. Think about how prescient Drew and the Navy's thought in 2018. That was way ahead. He goes, the 2 things that matter are the harness or the workflow engine that codifies your domain knowledge expertise. And the data layer, that persistent memory layer, that training data in particular. If you can master those 2 things and design the system to have models be disposable, you're going to win the AI race. And that actually underpinned the way I thought about the architecture at Horizon3. And so, for 6 years, we designed the system so the models were completely disposable and we could make use of the latest and greatest models. But more importantly, I don't need an LLM to tell me how to run an Nmap command. I've got domain experts that know how to code the Nmap command correctly.

Right.

Right. Just code the goddamn command correctly. Right? Just go do it. So there are parts of the problem where good old-fashioned machine learning, reinforcement learning, or automation are more than sufficient. And there are other parts of the problem where you want to iterate with an LLM because you need a form of creativity or classification or so on. Like LLMs with enough context are really good at classifying the value of data, the value of a credential, the value of a host. And so on to help you prioritize. So when you think about an AI system under the covers, you design to have the models be disposable and use the right technique for the task with the right domain knowledge workflow layer and the right data context layer. The reason why I bring that up is the most crucial aspect of our architecture is this knowledge graph, this cyber terrain map. It is the long-term memory that, and context for all of the agents that interact with it within our system. And so that context layer is a graph. And so we can actually do extremely accurate prioritization through techniques like centrality analysis and classification and context building and so on, enrichments and whatnot, because we focus on that long-term memory layer first. All these other jokers that have started companies the last couple of years over-rotated towards the LLM is magic and sentient, and they just built wrappers to Claude. Earned a crap ton of money on tokens. They got nothing to show for it.

Right. A lot of them are getting investments though, too. So don't tell the VCs.

Oh, they got investment and then they handed it to Anthropic.

Exactly.

And now what's happened is there's a million ways to kill a company. One of the easiest ways to do it is you raise a crap ton of venture at a crazy high valuation and late stage investors expect you to grow revenue into that valuation. But you burned all your money on tokens and not hiring a sales team or build go-to-market.

Yeah.

You're screwed. Like there ain't no way you're gonna recover.

It's been throughput right into the AI companies. Yeah. And there's been a lot of that. I suspect some people got their piece along the way and did all right, but you're right. There's been a lot of that. And, you know, a lot of just, that's, I think some people have done it knowingly, I'm sure. And some have not, as you said, some just didn't understand what they were getting into cuz they hadn't run companies before or built them. And now they look up and they go, well, we don't actually have anything as it turns out. We were just kind of a tunnel and now all that money's gone and people are coming looking for it. We got problems on our hands. So I got one other technical question before we move on and wrap this thing up. So you talked about LLMs, you talked about this disposable concept and the models and the data structure, all that stuff's really important. I am curious about your thoughts on small language models. People don't talk about small language models at all, it appears, comparatively, right? It's straight to the LLMs. And like I said, AI is magic. And I'm seeing a lot of success in targeted small language models for specific purposes as being very efficient and effective. You could have a team of small language modeled agents that work together and they all just stay in their own damn lane and do their thing and get better results, it would appear, than trying to build these large language models that can do everything across the entire process. But I'm curious what you've seen in that space.

Yeah, so it's super interesting. There's actually a couple of areas to double-click on. So first and foremost, the effectiveness of a model is almost exclusively tied to the diversity of its training data. So if you have limited, shallow training data, your models are going to overfit. And they're going to perform like crap in the real world. Fraud and cyber are 2 use cases where you actually don't have a lot of diversity of training data. And you've got the enemy trying to deceive you or beat you. So actually, these models are not good. A lot of these AI models for cyber work really well in labs and ranges, and they absolutely suck in the real world. Because they can be, we actually proved this with Microsoft Defender. They've got a new AI agent in there that's supposed to do something magical. You can get the eye of Mordor to look way over here while you sneak in the other way. Like, you can induce the defensive AI model to look and get distracted, and then the attack can go off and, and do their thing. But there's a more interesting question. Let's talk about blue team agents again. We did some interesting research that said, what's better, a jack of all trades agent or a highly specialized fit for purpose agent? And it turned out the jack of all trades agent was awful. It kept getting distracted. It was like a teenage driver getting distracted on their phone and veering off the road.

Mm-hmm.

Super easy. And then they would overcommit towards a path, get super greedy, and then fall off the rails. What we found though was if you took an agent and said, your job, agent, is to only hammer nails. You are not allowed to do anything else except hammer nails. You're gonna be the best at hammering nails in the world. Guess what that agent could do? Hammer nails really well.

Hmm.

In AI terms, that's called constrained action spaces. You want to give an agent as constrained an action space as possible so it can do that one thing really, really well. So what that means in your blue team agent architecture is you want a domain-specific specialized agent just focused on fixing CrowdStrike EDR. You want another one that's just focused on fixing WAF or like a specific WAF vendor or a specific data DSPM thing or specific app type or so on and so forth. And in domain-specific agents operating with constrained action spaces dramatically outperformed the jack of all trades. So I think that's where you can see, I think it's less about small language models unless you are sure you've trained it on diverse data. I think it's going to be much more about constrained action spaces and fit-for-purpose agents that are generic agents. Like they're using OpenAI under the covers. But you've intentionally constrained what it can do to minimize hallucination and drift.

Well, that's interesting. So large language model, but constrained instead. So it's very, very focused as opposed to trying to make it focused by just having the model be smaller to begin with, which isn't enough data. And therefore it doesn't have the depth of knowledge to be as good. So listen, we're running out of time here. I know you have a hard stop, in fact, but I do have one closer question. Every guest gets this who's ever on the show. I don't know if you were prepped for this one or not. So you might be in a tougher spot than most. But the name of the show is Unspoken Security. And with that in mind, I need you to tell me something you've never told anybody before, something unspoken. Please don't confess to a crime. This is recorded and I'll be obligated to tell people. So if you've murdered somebody in high school or whatever, please don't tell me that. But I'm curious, you know, tell me something you never told anybody before, something unspoken.

I'll go personal for a minute. The inspiration for me to become an engineer that got me into tech was my dad. And my dad When I was young, 6 years old, he would buy toy robots on business trips. He'd come home and say, hey, here you go. And I'd turn it on, it wouldn't work. And he had somehow sabotaged, he had cut a resistor or did something else.

On purpose?

On purpose. He's like, well, I don't know why it doesn't work, but I'll tell you what, here's a soldering iron, a multimeter, and a screwdriver. Maybe you can figure it out. And at 6 years old, my dad taught me how to methodically kind of take this thing apart and troubleshoot and figure out what the problem was. And he institutionalized this idea of systems thinking, take a complex problem and break it down. 'Cause he was an electrical engineer, but he was a test engineer. His job was to break stuff and then methodically figure out how to fix it. So for me, like my dad passed about a year and a half ago and it, it still hurts. Like it still kills me. You know, he was the first person I'd call before a VC meeting or last person I called before I walk into a key meeting or board meeting, whatever. First person I call afterwards. And so it still hits me every time.

Yeah.

Because I still have it in my phone and, you know, still my favorites and so on. I could never get rid of it. So for me, it's the role that you play as a parent and how impactful you can be on somebody if you do things the right way. And so what I never told is, man, I hope I have that same impact on my own kids the way my dad had that impact on me. I think about him every day. I think about what he did for me every day. I think about the conditions that he created for me to become who I am.

Yeah.

And I just hope I can do the same, man. It's a lot of pressure as I think about it, but it's not something I talk about openly because I just hope I do it right.

Yeah. Well, I, first of all, thank you for sharing that. I get a lot of different things here and some are more touching than others just because that's the nature of people. I'm sure for what it's worth that you're doing a really good job of it because you're already thinking about it. And the fact that this is on your mind is evidence enough, right? If you're thinking that way, you're probably acting that way as well. I'm going to ignore the fact your father gave a 6-year-old a soldering iron. And pretend that part's not, not a bad idea. Well, times were different.

When I finally fix it, what's hilarious is I'd finally fix the toy and then it would turn on, like spin all its lights. I dropped the soldering iron on the carpet and show my dad and it would burn a big hole in it. So when we changed the carpet, my mom cut out the 12-inch by 12-inch square full of soldering iron holes as memory. So yeah, like giving a 6-year-old soldering iron notwithstanding, it was quite impactful.

It's a different time. We played with lawn darts then. We didn't ride with seatbelts. I remember riding in the car. My parents had the windows up. smoking cigarettes, just hotboxing us for 250-mile road trips. Like, times were different. But the parenting that you're describing is amazing. That's an awesome, sweet, fantastic story. And I'm sorry you lost him. I know how that feels. I've gone through a bunch of that myself over time, as we all do. And I'm sorry about that. But it's a great story and it's great that you've told it and others will take away from that now. So he's given to yet another group of people and I appreciate you sharing it. So is there any last thing you do want to say to the audience before we wrap? We have a couple minutes here. Is there anything left you want to say? Anything you want to push? Anything you want to talk about? The product, whatever it might be, real quick, if you want to plug anything before I kick you out of the room.

I'll kind of end with actually not a product push, but an observation that I have about the cyber industry. So ChatGPT made everybody look the same and sound the same. And it's super hard now to differentiate, like, what the hell does this vendor even do? And how did that surface at Black Hat? Everyone had a gimmick, man. Like you had comic book stores and candy factories and inflatables and Man, if I'm spending a week in the desert in stale cigarette smoke away from my family, I'm not here to get gimmicks. I want to talk to technical people and actually learn stuff. And so one of the things we took at Horizon 3 very seriously was ensuring we were always technically authentic. And it starts with me and goes all the way down. So I think we were one of the few people at the expo that actually staffed our booth with hardcore technical nerds that loved talking tech. And so I hope honestly, like, We don't lose that technical authenticity as a community, as an industry, because man, the vendors are just getting out of control. And like, if I'm going to take time away from my family, I'm going to go talk to a nerd and learn something. And I hope that we start holding people accountable for the gimmicks and the absurdities and go back to basics of like actually working on cool stuff. I love that.

Yeah, I don't need another person trying to sell me fake candy bars or dressed up as a clown or whatever's going on. These multimillion-dollar booths that are all flash and very fancy. I love it. You know, marketing is having a really good time, but Yeah, I'd rather talk to experts, whether it's tactical experts, whether it's intel experts. I want to talk to people who know how to make us all safer because that's the job. It's not about the next art project that somebody's putting together for their booth. So I appreciate that. Listen, I appreciate you coming on, Snehal, more than you know, frankly. This was a fantastic conversation. I really appreciate it. I'm excited to see what happens with Horizon 3. I'm a big fan of what you guys are doing. I look forward to seeing more of that. Thank you for taking the time. And for anybody out there listening and watching, thank you as well for taking the time. If you like the show, please like it and Share it with others. Subscribe if you haven't yet. If you don't like the show, don't tell anybody. Just shut up. But tell me, and I'll see what I can do to try to make it better. If you know somebody who should be on the show, let me know. I'm always looking for new, exciting guests. But I'm going to go ahead and wrap here. So again, thank you, Stijn Hall. Thank you, everybody out there. With that, I'm going to close it out. This has been another episode of Unspoken Security. That's a wrap for this episode of Unspoken Security. If you enjoyed this episode, follow us wherever you listen to your favorite podcasts. See you next time.