Black Hat 2026: Why Threat Researchers Are Hoarding Zero-Days | Casey Ellis, Bugcrowd

A lot of people are just sitting on zero-day now.

This 23-year-old vulnerability, which is costing $10,000 today, would be probably $1,000, $100. Would you pay for that?

And the other side of the question is, what would an attacker pay for it? Um, in reality, this little guy behind me is tearing apart firmware with 27 billion parameter models.

I mean, it's no longer the CEO is going to click on a phishing link, but my Bob in accounting is going to build an app just because he's being put pressure on by the boss to use more AI.

Unleashing this stuff wholesale across the internet is going to cause some serious bumps in the night. It's a massive level up if you are technical, but it's designed for people that aren't technical to start to do more technical things. The vuln-pocalypse isn't here yet. Like, we're in the slop-demic right now.

One of the biggest cybersecurity conferences of the year was Black Hat 2026. And in this week's episode, I have Casey Ellis, who has been a founder of Bugcrowd and been the movement behind disclosure.io. I wanted to talk about what some of the themes that he noticed, especially because he works a lot closely with the offensive security community through the work he's been doing with Bug Bounty and other initiatives he's been working So in this episode of AI Security Podcast, we talk about some of the themes we saw at Black Hat and another cybersecurity conference called RSA that happened in SF earlier this year. What was the change in the mindset of how security leaders and practitioners who were on the ground feeling about the AI future? What were some of the changes in conversation? What were some of the themes that we noticed in terms of vulnerability management, exposure management as an ecosystem as it was evolving? And we also led down the path of going into what should organizations be looking into, especially if you have in the supposed vulnerability apocalypse and a lot more. All that and a lot more in this episode of AI Security Podcast. If you have been enjoying episodes of the podcast for some time and are here for a second or third time, or maybe more, I really appreciate if you take a quick second to hit the subscribe or follow button, whichever podcast platform you may be listening or watching this on. We are on Apple, Spotify, YouTube, LinkedIn, and wherever you consume your podcasts from. I hope you enjoy this episode with Casey, and I'll talk to you soon. Peace. Hello, welcome to another episode of AI Security Podcast. I've got Casey with me, a longtime friend, man. Thanks so much for coming in.

Thanks for having me, man.

Maybe just for the audience who may not have been familiar with your work in the past, if you can share a bit about yourselves, your professional background as well, man.

Yeah, sure. Uh, geez. Um, I was born at a young age is generally how I start that one. Uh, No, look, I've been in security really since high school, sort of old school hacking days, transitioned across into the practitioner side pretty much straight away, and then worked on the tools for a chunk of time before I cut across into the solutions architecture and kind of more businessy end of things. And I got it in my head I wanted to be an entrepreneur, so kind of broke bad at that point. And, you know, long story short, that led to me starting a company called Bugcrowd. back in 2012 was when we kind of kicked off working on that. Bugcrowd didn't invent vulnerability disclosure or bug bounty programs, but we did sort of pioneer this idea of putting a platform in between the research community and all the people that kind of need their help. So that escalated. And in the middle of all of that, got pretty involved in policy as well. So the other thing that I work on a ton these days is a thing called the Disclose.io project. And that's really about, you know, normalizing security reporting, safe harbor on the legal side, and then the infrastructure to actually support it. across the internet. Because as we turn everything up to 11, there's definitely a growing need to do that. So that's the short version.

Fair. And I mean, and not that the accent doesn't give it away, but you're definitely very Australian. But this happened to be— the Americanism hasn't caught up to you yet.

It's definitely rounded off. It's funny because when I spend time in Australia and if I do something like this, I'll listen back to myself and I'm like, oh, there it is. You can kind of hear the The twang sort of build back up. So it's curved off over the years talking to Americans. I actually live in the Bay Area, but still have deeper roots back home as well.

Yeah, fair. And maybe talking about deep roots as well, one of the things that has been core to our cybersecurity ecosystem has been the Black Hat conference and RSA conference. Obviously, we're all attending those conferences to get to know what the top of mind things for CISOs and practitioners are. But I'm curious because I think you and I have both seen RSA and Black Hat. Was there an obvious theme in terms of whether it was related to the AI? I already said AI. I should not have started with AI, but I already said AI. So, let's just say, what theme that came across that RSA and Black Hat that stood out for you, obviously seeing both the conferences?

Yeah, look, I mean, the theme through both was definitely You know, AI kind of is a standalone thing that we're all trying to wrap our heads around. I think this year, and it actually applies in a lot of ways to besides Las Vegas DEF CON and some of the more, call it skeptical and like practitioner heavy cons as well, we've kind of all reached a point where it's like, yeah, this is a real thing.

Yeah.

Because I do feel like the security industry just in general, I mean, we've got a natural allergy to hype and the hype cycle. Around AI, I think turned a lot of people off. I do think that hype cycle was real and there was a lot of stuff about it that was kind of annoying, but at the same time it's like, no, this is a real, this is a real thing that's gonna eventually touch and change everything. So I feel like this year between RSA and Black Hat, like everyone's basically on that same page. I think there's definitely a difference between how vendors are talking about things and how the practitioners are talking about things. You know, CISOs all the way down and even like policy folk. All the way down, you know, the vendor halls, like, here's this new solution that solves absolutely everything because we've got AI to do that now. And like the reality of that on the coalface is maybe not quite as rose-colored as it might get presented, not to put too fine a point on it. And there are vendors that are actually trying to do this really well too. So like massive shout out to those folk. But I think just as a general observation, there's a lot of, there's a lot of sort of shouting over the top of each other about how, you know, agentic your thing is. And, and that's kind of the main selling point at this point. Which to me kind of misses the mark, but it's definitely a common theme. Yeah. Last, last thing is like, I do think, yeah, the, the big thing about Black Hat this year just was the way, like, literally the entire Mandalay was, was booked out. And I think it's, you know, it's the first year I've seen them actually do that. So, so the, the whole kind of opportunity for hallway con and community and, and the kind of stuff that a lot of us actually attend Black Hat for. you know, alongside the talks and the booths and all that kind of stuff. There really wasn't a lot of opportunity for that in the venue. So I feel like sort of Black Hat's trying to be more like RSA in terms of commercializing the conference itself. And maybe they jumped the shark a little bit on that side. I heard that from a lot of other people as well, but that was, that was definitely a difference between the two.

I think worthwhile calling out to what you said. I think the events that I hosted for the book launch that I had was all either in Venetian, Cosmopolitan, definitely not in the Mandalay Hotel, which is I mean, it's just the venue technically, but everyone was traveling between hotels just to do what you said, because everything was booked out for whatever event. So you almost were running out of places to have hallway conversations and everyone just talking, or at least there's like a mob of people walking together. I was going to say something that I thought was interesting, and in terms of conversations that I had, was from an RSA perspective, I noticed that There was this, if I were to step like a 10,000-foot view of RSA, the conversations that were there, there was a lot of focus on visibility, shadow AI, AI agents were kind of sprinkling, but there was sprinkles of, hey, we can do SOC automation in terms of product ecosystem. In terms of CISOs and practitioners, I felt they were still trying to get their head around. And if anyone who had not given even a remote serious thought to AI and how that would, what that would mean. They were still on that cusp of, oh shit, I need to kind of at least get a handle of this because people are moving forward. And fast forward to Black Hat, which technically is like 6 months, I guess, between RSA. I found that many people had focused on the whole coding agent pieces. Then they had conversations around the— they were a lot more mature in terms of, hey, we need to have an agent policy.

Yeah.

The AI governance conversations were a lot more mature. People were talking a lot more about how can I automate my SOC, my GRC, and bug bounty, which is kind of, I know your wheelhouse, like you clearly have spent a lot of time in there as well. There's a lot more conversation around how, and maybe because of the Mythos effect in between as well, there's a lot more conversation around, oh, what does it mean for runtime? What does it mean for how hackers would misuse the technology, even if it's open source?

It did.

Anything that, I mean, obviously this is what I noticed, but I'm just curious if any of those—

There's a heck of a laundry list. I can kind of agree with basically all of that. Yeah, look, I think it was funny because I wrote a blog post just before RSA talking about where I predicted agentic AI, like the intersection between that and offensive security. Whether it's the good version of it or the attacker version of it, kind of where all that's going to go. And just calling out really based on my experience of hacking the internet at scale for the last 15 years, it's like we're actually still in a pretty bad spot when it comes to vulnerabilities and opportunities. And that's not necessarily a concept that's well understood, or where it's understood, it's not necessarily talked about or accepted. And then literally 2 weeks later, Mythos drops. And I think that was really useful as a, like, to me, that was almost like a, you know, a Snowden moment in terms of like everyone suddenly thinking about this cybersecurity thing all at once. I think, you know, Mythos and Daybreak did that, you know, right across the board in terms of AI's ability to speed up finding things. But to me, like, the underlying thing is like, no, there's a lot to find. We're actually still pretty bad at a lot of this stuff. And if attackers are sufficiently resourced and motivated, they'll, they'll, you know, nature finds a way in a lot of ways. So I think that was a way more, it was kind of more of a given, I guess, at Black Hat. Like, I felt like that conversation at RSA, I had to do some, I had to put some sort of evangelistic weight behind it. I didn't, I didn't really need to do that at Black Hat.

Was there something that, in terms of all the conversations you had with the bug bounty folks and others in the offensive security side, Was there, I guess where I'm going with this is that I'm curious in terms of the usage of AI in that ecosystem as well.

Sure.

That's usually the people who are in a way driving the automated pen testing, the runtime assessment, like all the, insert the, to your point, the laundry list of things that can happen on the right-hand side of the SDLC. Was there any theme that stood out there in terms of their maturity or what you saw in terms of how they're— Yeah.

Yeah. And probably, you know, the way I'd phrase that is that like, I've been, it's like bug bounty to me is sort of one expression of this that people have gotten used to typecasting into like web vulnerabilities that you can see from the outside. Yeah. My angle on offensive security is it's the entire spectrum. It's not just that. So when I'm talking to people about the role that AI plays, it's across that entire spectrum, not just the web part.

Yeah.

But what I'd say is that it's a pretty broad spectrum of, I guess, responses to it. Like everyone's using AI in some way to improve, AI is a tool, like it's reducing the time to effectiveness. Like everyone's pretty much on that boat at this point. And for the better part, that's been true, I'd say for 2 or 3 years now.

Yeah.

It's just now that you can kind of talk about it and people don't think you're trying to you know, shill OpenAI or Anthropic in the process. It's like, oh, that's actually a real thing. So there's definitely that. I do think that there's a, you know, there's a lot of uncertainty. Like I heard a lot of folk talking about, you know, if my special thing's all of a sudden getting commoditized by technology, then what does that mean for me in the future from a career standpoint and all that kind of stuff? There's a lot of that in the same way that I think there is in a lot of different industries right now. But that was sort of like, A bit of a light bulb moment for parts of the community that are concerned about that. And yeah, probably the last thing I'd say there, and this is like a big takeaway, is that a lot of people are just sitting on zero-day now. Because the thing is that like triage queues right across the board, not just vulnerability intake from the outside world, but like the SOC queues, the, like everything's just gotten dialed up from a noise standpoint over the past 12 to 18 months at the very least. And that's causing lag. So if you're, if you're on the outside trying to help, you know, be it in context of a bug bounty program through like a HackerOne or Bugcrowd, whoever doing it directly, or you're just trying to report a vulnerability to get it fixed, that's actually pretty hard to do right now. A lot of folk are just basically not hoarding zero-day for like, you know, they're not like thinking about breaking bad or doing whatever. It's just too hard. So they're just sort of sitting on it.

So do you find that the— it's an interesting thing, right? Because a lot of that— because is the volume AI slop? Because obviously I'll put some color to this as well, because you know how you and I have seen this world where the whole ecosystem of, hey, I have a disclosure program came in also from the fact that I'm sure, as I say this, most people relate to this. We get emails saying, hey, I have a zero-day I've identified. I have a whatever, pay me $5,000, $10,000 a week.

Big bounty and extortion and all that kind of stuff.

Yeah.

Yeah. Yeah.

Is this like a volume of that has increased or is it legit, or is it just disappearing?

This is the problem is it's actually both. It's actually both. So what's happened is the cost of discovery, or like the cost of getting to a point of discovery and a point of weaponization to where you can create impact has gone down, especially for competent operators, but like right across the board as well for the more trivial types of vulnerabilities. So there's that. Then like the, you must be this tall to ride bar has gone down. So there's more people jumping in and actually doing that. And at the same time, the cost of actually writing a submission and sending it has gone down as well. So you've got like this increase in noise, this increase in opportunism. I think a lot of it actually is people that aren't necessarily trying to be extortionate or anything like that. They're just like a 3 out of 10 for usefulness and like an 11 out of 10 for enthusiasm. And they've got new toys, right? And that's always been a thing. Like, we've seen that, you know, versions of that the entire lifespan of Bugcrowd. That's not— that's like the community just does community things and you gotta figure out what to do with that.

Yeah.

But the problem is if that noise level gets up too high, you end up in a position where you're spending so much time dealing with it that you miss the stuff that actually matters in the process. And the volume of that is going up at the same time.

Yeah, yeah. I think I was on a call with a CISO and I think I remember telling him kind of similar vein around volume of notifications coming in has increased, but I think they were almost at the point where if the email had something to do with remotely, I have found something without any evidence, it's ignored instantly, even though it may be a genuine one.

Yeah.

Because obviously, understandably so, you don't put all the information in the first email clearly, but then they said the volume was so high. That they were choosing to just ignore the ones that were coming with not much evidence, just say, hey, pay me money. Or even the fact that I have found something legit. And it's usually people were finding success from, I reach out to say Casey, who's working for a company and I'd reach out to them personally and go, hey, I think I've found something here with probably you guys should face kind of, is that, have you found a better way? Because that obviously is an extreme scenario.

Yeah. Just turn everything off and go back to the abacus. I think it's probably the solution.

I feel like you turn it off and hope for, just reach out to the right person.

Look, yeah, in terms of better ways, there is definitely an element like of researcher education around this, like helping people get more efficient, I guess, at delivering their point when they're trying to submit a vulnerability report. Again, that's something that we've always tried to do, that I've always tried to do in a Bugcrowd context, because that's always been a need, but now that need's on A lot of Adderall and then get a million miles an hour. So like, there's definitely an increased need for that on the community side, just in general. I think downstream, you know, when I'm talking to companies about how to think about this, like you've got to make sure, to me, the biggest thing is making sure that your policies and kind of your expectation setting to the outside world is really clear. Because, you know, if you do that, then you can end up in a position where if there's a disagreement at some point, you can just point to the thing that you've already said. And oftentimes that helps to to deescalate. And there's a lot of need for that right now. The other is just understanding what your risk model is. You know, like the big thing, this is probably the other big observation from RSIN Black Hat, is this whole concept of like vulnerability management sort of being, you know, a bit of a fool's errand if your goal is to make sure that everything's fixed. It's like, we're not going to get around to all of it. There's too much. Like, we've got to actually think about, You know, what our threat model is, what are we trying to prevent? Like, what kind of attackers are we expecting? Like, what's their level of like economically rational motivation to do the thing? How do we sort of reverse out the things that are most important and make sure that we're not missing that stuff if it comes in on the queue? And then kind of work backwards from that instead of trying to eat the whole elephant and figure out which part was best.

There's a few things to what you said as well, right? Do you find people even understand How to measure risk that AI is bringing? Because I know the— and I don't know if you noticed this thing in the work you've been doing, or at least the conversation you've been having across America, but I definitely found that between the conversations that I have in America versus Europe and UK have doubled down on the whole mythos readiness assessment. And the Big 4 companies keep selling them the idea. They pay 6 figures just to get that assessment. You're like, I don't even need this. But It's like, and I guess goes back to the maturity of these things as well. Was there more of a mindset of that, hey, Mythos is actually bad, or was it Mythos is actually good for the industry? Because I'm curious also, because to your point, there's the community perspective, and then there is the reality of if this hits all the vulnerabilities we spoke about earlier, which we could never patch before AI, but now we are with Mythos. What was your—

and now more people are capable of doing that potentially, so we can't necessarily predict intent in the same way.

Yeah. Yeah.

I think in general, the US is a little ways behind on that, frankly. Like a lot of the thinking on that stuff is very reactive. Like even talking about it as a mythos problem to me kind of misses the point a little bit because like I've seen, literally I've seen people do this type of stuff. with open weight models and with unrestrained frontier models for the last 3.5 to 4 years. Like, this is not a new, it's not a new thing. It's just that everyone's talking about it all at the same time now. So there's that. Like, it's not sort of like this didn't suddenly happen. We just all suddenly started talking about it. And I don't think that necessarily registers with folk. But I also think that, you know, I was in DC actually at the White House, like the week that the export control stuff Happened. Yeah. And like, without going too far off-piste, I do think a part of that reaction was a whole bunch of policymakers suddenly realizing that this is possible in the first place.

Yeah.

If that made sense. And in reality, a lot of people have been trying to say that and kind of get them onto that page for a while, but this was kind of their moment where they're like, oh, it's real. So there's this sort of reaction that happens to that. And that's been, I think that's driven a lot of thinking around how this is going to get solved in ways that I actually don't think are necessarily correct, if that makes sense. You're not going to export control your way to a safer security, it's one safer security state internet-wide.

Yeah.

You know, at the same time as I think kind of unleashing this stuff wholesale across the internet's going to cause some pretty serious bumps in the night. That's going to happen anyway because the frontier stuff's trailing, you know, 3 to 6 months behind. And yeah, and everyone's getting their hands on that nowadays. So it's one of those ones where it's like, we don't really, I think we keep on in general talking about this sort of problem space as though we've got the ability to control it, whereas in reality we actually don't. Like we can control what attackers will be capable of when they show up with these tools. To treat it as a thing where we can tell the internet like, No, don't do that, please. And how do you— Yeah, like that's historically, you know, I've, I've especially doing the bug bounty and the vulnerability disclosure thing for so long. Like if there's one thing I've learned is that, that, that does not work. Like the internet isn't listening.

Yeah. I think someone said this. I can't remember the name of the person, but at Black Hat, someone said, Mythos is the worst of the AI model today compared to what we will see in the future.

Yeah, I agree with that.

Because you almost look at this going, we are talking so much about Mythos and everything else around it. But, and as much as I appreciate the marketing behind it, now my parents know what I finally do and what my book was about.

So it has been used for that. Like the marketing problem and actually driving awareness around this, I think that's an underrated problem to solve that it did actually solve in some ways. But sorry, I cut you off there.

No, no, no, but you're right because it definitely made everyone Know what cybersecurity is, know what Mythos is. It was there everywhere, and great job on that. So keeping that marketing thing aside, I definitely found that what people kind of took away from the fact that it— this is, since this is like the worst model that we can get today and the future models are going to be better. And I think there was another argument about the fact that, oh, it's going to cost me $10,000 to find that 23-year-old vulnerability. But then the counterargument to that, to someone calling out, was the fact that today, when a frontier model goes through one of those benchmarks that people look at, if you compare the first few models to the models that are being used today, the amount of tokens used is actually far less.

Yeah.

Which means it's actually costing you way considerably less than what it used to run a benchmark, which means this 23andMe vulnerability, which is costing $10,000 today, would be probably $1,000, $100. Would you pay for that? I mean, compared to what you can find in your organization?

Yeah. Well, that's one side of the question. The other side of the question is what would an attacker pay for it in reality? And this, by the way, is why I roll my eyes when people say bug bounty is dead. Because I'm like, if you think that's true, then you don't actually understand what a bounty is. To me, it's a function of the fact that like every vulnerability and every chain of impact that can be created, Has some sort of value to it, to someone. And it, the whole thing kind of functions as a marketplace that, that value might be zero legitimately, or it could be really high. You've gotta actually figure out what that is and make sure that you're getting information to the right place. So yeah, it's, it's one of those things where, I mean, again, like this, this little guy behind me is tearing apart firmware with, with 27 billion parameter models and a harness that's actually optimized for that particular task. And it's, it's effective.

Yeah.

Right. So it's not, I don't need Mythos or a Frontier class model with all the context memory and all those other things to actually do that. It's just more for me actually knowing the thing that I want to do from a research and an output standpoint and optimizing AI pipelines to achieve that outcome. There's again, like that's been a thing now for 3 or 4 years amongst a lot of the true VR and offensive community. And the Frontier version of it is just It's sort of put it on easy mode because you can just throw something up into Daybreak or Mythos and say, hey, pwn this thing and it'll come out with something that mostly works.

Yeah. And I think what you said earlier, which was really interesting, that you have to be yay high to take the ride. Well, as a script kiddie who may be watching and listening, or maybe looking at these conversations, oh, Mythos probably is not accessible today, but it would be on an open source, open wait model soon. So it's only a matter of time. Is the skill set for the entry— and going back, using that same analogy a bit further, is the bar to entry to find vulnerabilities low for everyone? Is that just a theme that you saw? And when I say everyone, obviously I'll put an asterisk in it. Obviously not non-technical people. I'll keep them aside for a second.

Mm-hmm.

But if I was— I'm a recovering malware analysis person. I tried that for one hot minute and I gave up. Very quickly. And I've been told multiple times by people that I should pick up that skill set again, because I'll find it way more easier. And I wonder how much truth is there to that, especially after you've had the Black Hat conversation as well.

Yeah, that's a great, that's a great question. I think on the offensive side, yeah, there is definitely a difference between vulnerability discovery and like exploitation. And, you know, this is actually, you know, when I say like the vulnpocalypse isn't here yet, like we're in the slopdemic right now, that's a part of what I mean. Because like folk can find, it is easy to find, that's a symptom of vulnerability. I'm going to test that and prove that it's real. Job done, right? Like that part's actually pretty easy, but attackers and exploits don't function as a list. They function as a graph that has a starting point and then impact endpoint. And usually what it comes down to is that traversing that graph is very much up to the skill and the context of the operator. So yeah, I think, you know, if you're talking about just point and pwn, you can definitely do that and have like simpler issues fall out, I think, in a bug bounty context. And even in like an adversarial attacker context, the challenge you're going to have with that type of thing these days is that it's contested. Like it's actually really easy to find that. So you're going to dupe out if you're a bounty hunter or end up bumping into a competing adversary if you're an actual bad guy. When it comes to doing things that are more sophisticated, I do think it's really, since I picked up, put the suit on and started doing the business thing, I've tried to keep some of my skills up from the old days, but the older I get, the better I was. What I've found personally is being able to jump in and actually steer these models towards an outcome, with more of like a product management level understanding of how this all works in contexts that I'm not as familiar with or a bit rusty on, like that works really well. Like I can guide stuff to an outcome 'cause I understand how the operator mindset works.

Yeah.

Even if I don't necessarily understand the bits and bytes of how to actually do it myself. So people that have that kind of knowledge or that kind of intelligence, I think they can become really effective with this. Like the folks that are doing like true VR, like the scary stuff, they've just, upleveled their game 10 to 100x because they're not, they're not like delegating their intelligence to the model. They're using it as this like super partner, like what's the word? The wingman type thing.

Oh yeah. I mean, glad you didn't say copilot. That would have just been—

I was trying not to say copilot.

So I had to find another word. I know. It's been butchered so much. You almost feel like it's like, I need to come up with a better word than a copilot because they just have not done justice to that word. But loyal wingman is Yeah, that's actually a wingman or women who doesn't leave you anywhere.

It's a military hardware term, but I think it actually kind of applies a lot to this type of thing.

So yeah, 100%. And I think you've kind of— what I think based on what we are seeing, at least what you saw and what I saw, it definitely seems like a theme that the offensive side has definitely leveled up quite a bit in terms of between RSA and now. And there's people obviously Being able to use much more of their understanding of how this operator model works to a large extent. I'm curious more from the security practitioner, like the defense side as well, how are you seeing them address some of these? Like essentially, we're not even at the apocalypse yet. However, and we're all getting the volume of increase in reports coming out from external sources, internal sources. The AppSec pipeline seems to be quite full because there's a lot more code being created than it was before because everyone's using coding agents. The cloud build, if I, it feels like, or at least to me it felt like everyone I spoke to, they all said there's a huge volume everywhere across security. And I'm curious if you kind of saw something similar and was there things that you noticed that people were someone doing better or you heard stories that were People could maybe learn something from as well.

Yeah, no, for sure. I, yes, 100%. And, you know, I kind of mentioned that a little bit earlier with the observations from RSA. There was definitely more of that this year. Like everyone's kind of strained really is what it comes down to. I think that like the folk, you know, something that I'd started talking about probably 2 years ago is that like the defender's dilemma always was a real thing. I feel like the thing that we've transitioned into now is that no one's really able to argue with that anymore. Because like, I do think, you know, there was, it's up for healthy debate in a lot of ways, but to me, that's an economics problem that you either understand or you don't. Right. I feel like AI has kind of highlighted the fact that's a real thing.

Yeah.

You know, something that I was talking about a while back is the idea that like, we're used to this sort of attacker-defender innovation, like cycle time. In the cybersecurity industry. And what AI does is that it squishes that like OODA loop to the point where, you know, maybe we're not going to fit into it for too much longer. So at that point, we've got to think about like, how are we actually changing the game here? Like, how do we impose cost on attackers? How do we, you know, think very carefully about detection, you know, ejection, recovery, like all those different things moving into a into an assumed breach posture, continuing to work on prevention and defense. I'm not saying give up on that necessarily, but it's like, okay, this is going to fail at some point. So like, how do we actually, you know, in a resilient and an antifragile way, prepare for that? I think the folk that I feel are most on top of this are the ones that are kind of leaning into that mode of thinking. I know the labs are definitely putting a ton of work into thinking through How to support the industry in doing that in general. And it's still, I don't feel like there are necessarily clear answers to a lot of those questions to say, but that to me is sort of the set of the sail at this point.

Yeah.

Yeah.

And I think well put together because I think I agree on frontier models trying to make some impact there because they realize they can't do this on their own. We had a conversation and I think it's going to come out soon, which is the state of AI security conversation we had. The OpenAI folks, actually Clint Gibbler.

I don't know if you know him already.

Yeah, he works for OpenAI now. So him, we had Forrester analyst, Ali Mellon. We spoke about, funny enough, the same theme as to why our frontier models are suddenly partnering with all these cybersecurity companies or Accentures of the world or KPMG, whatever other consulting companies, the big ones.

Yeah.

One thing that I definitely found, and I love the dilemma conversation as well, because And this is a theme that I saw, which I definitely know people are seeing it, but it was very interesting. The dilemma kind of amplifies a bit more because these days, if you used to work for a bank, which has been there for 10, 15, 20, maybe 40 years, they've gone through the transition from your data centers to cloud, now going from cloud to AI.

Yep.

There are essentially 3 kinds of applications that I found that most people were talking about. Fully AI native application. And now that, that, you know, that many in there and a lot primarily are startups, I want to say, or they are companies that are just, they've been broken out of the traditional company. I think there's a few examples in Australia and the US for this as well, where companies have branched out going, hey, we are too rigid to change, but why don't we just start this side company? And that's like first bucket of defense that people look at.

It's like your digital transformation.

Yeah, yeah, literally same thing.

Same sort of thing, but for AI. Yeah.

Yeah, yeah. And that's like my— I'm looking at this as my first bucket of defense, where a lot of people have gone down that path, where, how do I do security AI-native way? And then the second one was the AI bolt-on, as I like to call it, where, hey, I'm a traditional enterprise, but I have my developers using coding agents, producing more code. I'm basically putting pressure through the neck of every employee that I can find, hey, use more AI, use more AI, but security, you have a $200 per month budget. Like, do whatever you can, but in $200. So there is that. So there is a dilemma more also in the context of they've been given access to AI. And whereas to what we've been talking about so far, the attacker is willing to pay $10,000 for a bounty or for a vulnerability. And then there is the third theme, which is kind of in that shadow AI bucket as well, which is the whole white coding becoming a norm, especially in the AI-forward companies where—

Yeah.

The non-technical people have started doing, building applications, pushing code into GitHub, and ties in really nicely to what you just said about the assume breach mentality, which traditionally used to have the lens of that, hey, it's technical people. Now it's like, it's like everyone in the organization.

Yeah.

It's like even this, I mean, it's no longer the CEO is going to click on a phishing link, but my Bob in accounting is going to build an app. Just because he's being put pressure on by the boss to use more AI. So he's like, okay, I'm going to use another application and build one application for myself and show it to my boss. Hey, this is what I made with AI. Look at my productivity. I don't know. I feel like it— do you— has there been a theme like that as well that came out for you in terms of how people are using AI? Has that changed in organizations?

So the Bob from accounting example, like I'm Close with the, uh, the folk that run the NOC and the SOC at Black Hat.

Oh yeah.

And, and at DEF CON as well. And, um, there was, there were a couple of things that they saw. One of them was they were actually seeing zero-day across the wire, which is not normally something that happens at that conference because—

Oh, wow.

Yeah. If you're sitting on something like, you know, that if you shoot it across the network at Black Hat, it's going to get caught. So like traditionally folk with that kind of tradecraft would just kind of keep it to themselves. That particular event. And that shifted this year, I think, because you've got folks that have capability but don't necessarily understand that tradecraft component. So that was one. But then the other 2, which sort of goes more to what you're talking about, is that like Port 3000 was pretty wild. You've got folks that have—

Yeah, yeah, yeah.

And they've forgotten to not bind it to any, any. And There were people that were sort of sitting around waiting for that to happen. So there was some interesting action on that side. And then the idea of like Bob from accounting who's, you know, vibe coded up or someone from the sales team or like people that are just there to do work, right? And they've, you know, what they're doing is actually kind of a good thing. They've kind of used AI to try to make themselves more efficient at their job. But in the process, they're sending, you know, prospect lists in clear text and like just all sorts of Silly shit like that. So, so like, that's a real, that is 100% a real thing. On the first version that you said, like, when we got, you know, one of the early aspects of the bug bounty story was when we, you know, started working with the Pentagon on Hack the Pentagon. And, you know, HackerOne got the initial thing and then it kind of blew out. But the part that's not necessarily talked about as much is that, you know, we were actually working with with them in the setup process of all of that. And the way that that happened, that kind of goes to what you're talking about, is it was through a group called the United States Digital Services Group, which was effectively the digital transformation arm for the DOD. So it was literally what you're just talking about. It's like, we are big and old and slow and organized. It's not even our fault necessarily. We might want to improve and innovate, but— Organizational inertia means it's going to be really hard. So we need this group that's like literally their job to disrupt the way that we think within the organization. And that was how that all kind of got moving when it came to introducing hackers into the Pentagon at that point in time. So like, I think that's going to be the right model for most larger organizations at this point, just because this is moving really quickly and there's an urgency to it. The other thing I would say real quick on that is that I do spend a lot of time, like a lot of the conversations that I have with senior leaders in particular, they aren't necessarily aware of what AI can be used for defensively or even from like a build standpoint because corporate policy has been like, don't use this stuff. Like the reality is that people are just going to use it anyway at this point.

Yeah.

Like relying on policy to prevent, that's probably not the best defensive thinking at this point. But then if you've got leadership that's not necessarily aware of the art of the possible, then you're going to end up with this pretty big gap between the folk leading the business and what your workforce is actually doing. So what I've been encouraging senior leaders to do is like, just use this stuff at home, like figure it out, like get a harness, like just play with it, figure out what you can use AI for to automate stuff that you're trying to do around the house. Think about your own home network security if you want to, whatever it is.

Yeah.

Actually get hands-on tools with using this stuff in an applied way and make sure you're at least comfortable with that. Because if you're not, you're going to end up with this huge gap between your office and your employees, and that's going to create this massive internal threat problem, basically, at that point.

Yeah. And I think it's worthwhile calling out, you don't have to be really technical. I know you and I maybe come from a bias of being technical first, but you don't have to be technical to kind of walk that path, even if you're a non-technical CISO.

Yeah. Yep. I mean, this is designed, like, the stuff is designed for that. Like you said before, it's like, It's a massive level up if you are technical, but it's designed for people that aren't technical to start to do more technical things. And yeah, like you said, it's not about necessarily being an expert. Like the advice I give is just get comfortable with it. Just be in a position where you can have a conversation and you're not like thinking the other person's talking about science fiction or something that is going to happen, you know, 3 years into the future. It's like if you still think AI is stupid because it said that there were 7 Rs in the word strawberry 3 years ago, like you're doing it wrong at this point.

Yeah. I think the way I've been explaining to a lot of people is the fact that I use iPhone as an example. It was the first smartphone, but okay, I'm not going to claim the first smartphone, but it was one of the popular smartphones as far as the market is concerned.

Yeah.

As far as the market is concerned.

I'm sure there's so many Canadians that are going to get pretty mad about that, but that's okay.

Yeah. But the reason I use the iPhone example is that it's been decades of iPhone being out. And I, even today, no matter how technical I am, I'm pretty sure I'm just using 10 to 20% of what the actual capability of an iPhone is, just because I chose to be comfortable with how an iPhone works. And I think that's what we are asking people to get to. You don't have to be like, oh, I know the ins and outs of every AI and LLM. I'm a data scientist or whatever. Just enough to make it usable for things you want.

Yeah. Yes, 100%. I think that's right. And that sort of thinking was the kind of thinking that enabled BYOD.

Yeah.

Because that used to be a dirty word way back in the day, and all of a sudden it became normal. And like everyone who was behind the curve on that had to play catch up at that point. Like this is another iteration of a similar sort of phase shift, I think, in how we do a lot of stuff. I mean, the other side of it, you know, the other sort of suggestion, this is something that I've actually been doing a lot, is like literally getting with friends and saying, hey, like you're building a thing or you run a small business or whatever else, like how can we find ways to use AI to make life suck less in the places where it sucks and actually enable you to do more of what you're like uniquely good at or what you're trying to do as a business or an operator. And just like, it's valuable for them, but for me, the act of repeating that kind of translation process is actually something that keeps me sharp on that side of things as well. So I think that's a really, that's just useful. And it's kind of fun. Do you know what I mean? Like being able to build silly crap is, you can just do things now in that sense. And I think for folks that are in a leadership position to actually have their hands on the keyboard with some of this stuff, I think is really important.

Yeah. Dig into the curiosity mindset that we always had. You just need to dig back into it. Exactly. It's funny, kind of similar reason why, so we've been running these free sessions on a fortnightly basis called AI Security Lab to what you said. There's an obvious gap. Even in Black Hat, the conversations were, I don't know what security workflows to start with. Or so we've had people from Snap, Adobe, I think we've got SpaceX coming in.

Nice.

The whole idea has been, it's a free session for 1.5 hours. Someone just shows you a workflow from end to end. Hey, I think the first one was about how do I do authentication for multiple, when I'm using a multi-AI agent chain. The second one was about how to build an AI harness that you can use with security. There's plenty of ideas, just, you can obviously make your own, but AI Security Lab, if people are interested. But the thing that you mentioned earlier, which I would love to go back to, is the gap that you've— I'm curious because you've been on the operator seat as well as on the founding seat as well. I'm curious, what is the gap that you are noticing in this ecosystem today that not many people either are talking about or it's not being covered?

Uh, on the, on the operator side, you mean?

I mean, even from a product perspective, it could be any, any, just pick any side you like. I'm just curious as to overall, considering you talk to both sides like me, I'm just curious as to what are you finding as gaps?

I think, yeah, I'm, look, probably the biggest thing is that there's just a, a, a norm, like a vast spread of, um, you know, understanding of the art of the possible, like acceptance of, of the technology and even thinking about like, where is this actually going to fit into things into the future? The spread of that is just insane. Like it goes from people that are like, you know, fully, I hate the term AI or AGI pilled.

Yeah. AGI pilled. That was actually, that was one theme as well. No one was saying AI pilled. Everyone said AGI pilled at BlackRock.

Yeah. Yeah. And it's, that's a, I can get on a long rant. To me, like the whole thing is like, no, I'm human pilled. I'm not AGI pilled. Like I believe in AI being as transformative as, as you know, like the internet in like 1998 kind of thing. Like, I believe that part.

Yeah.

The whole idea of like it being, you know, the purpose of all of it, I don't buy into that bit.

Yeah.

Like the whole idea of like folks that fully kind of accept the fact that this is a phase shift in technology and they're really leaning into that and they're thinking about it through that lens today. And when it comes to seeing around corners, like that's to me like right up the sophisticated end. And then you've got folks that are like, What I said just before, like it told me that there were 7 Rs in the word strawberry one time, so I think this whole thing's BS. That spread is so obvious, I think, at this point in time. And what it does is it creates all these potential impedance mismatches for an operator, right? You've got different levels of technical nativity inside an organization, and as a leader, Like you've gotta make your own decisions around where you think this is gonna go and then try to bring everyone else up to speed with that. And I think that's, that job's actually getting harder and harder as time goes by, 'cause that spread to me feels like it's getting broader, right?

Mm-hmm.

So that's probably, that's a big one on the operator side. On the vendor side, like I do think I'm in like regroup, you know, and think about startup mode. So I can ideate on this, but there's some stuff I have to keep in the pocket. A little bit, but I think in terms of like the problems that we're solving, like the whole, like there's so many agentic pen test vendors, you know, on the floor at Black Hat, for example. And it's like the pen testing was effective to a certain point before we turbocharged it and made it available 24/7 and got robots to do it. Like now we're just sort of dialing up the same thing that has been somewhat effective, but I wouldn't say super effective in terms of the overall risk posture of organizations over the years and just doing more of it because that's kind of what the market says we should do. Basically inverting that thinking and seeing what falls out is kind of a lot of what I've been doing. And I do think about it in terms of like, what are the bad guys up to? How do we reduce, you know, or how do we increase the cost of attack in ways that are rational, you know, legal, obviously all that kind of stuff. But how do we think about it more as a, The entire reason we're doing this is to basically frustrate an active adversary. So like, how do we, how do we, have we, where are the places that we've forgotten that? And how do we actually start to think about solutions that sort of orient themselves in that direction? Because like AppSec's awesome, shift left's awesome. Like those things are all good and important from a resilience standpoint. But when you come back to the fundamental of why we're doing this in the first place, it's because of the bad guy. So, all right, now what? Yeah, I think that's a thing that I see some, like a small number of vendors really leaning into, like deception tech, I think is going to have a really good time over the next period because like shout out to Haroon and those guys who've been like literally just quietly grinding away on that as a good solution for 12 or 13 years. And all of a sudden everyone's like, oh crap, we don't know what our agents are doing.

That's, yeah. I mean, it's actually AI deception companies quite a bit. I think we spoke to TraceBit recently, but yeah, to your point, deception is Finally finding its limelight.

It was always a good solution, but now it's sort of an obviously good solution, I think.

Yeah. Because I mean, it used to be hard to justify why do we have this thing which never lights up? What's the point of this thing that it never lights up? Isn't there a— I mean, and obviously we can go, dude, I'm just conscious that I only had you for an hour, so I want to make sure I respect your time for this as well.

I appreciate it. We could probably run on this for a long time, but yeah.

Yeah, no, we could have gone forever, but I, where can people find more about what you're working on and connect with you on all these things?

And yeah, for sure.

And I've got obviously more episodes, but where can people connect with you?

I appreciate that. So yeah, I write stuff at cje.io. That's probably the main kind of presence that I have on the internet. The Disclose.io project is at disclose.io. Yeah. Check out Bugcrowd as well. Those are probably the 3 main places to go. To go looking. And, you know, I'm always ranting on LinkedIn and X, Twitter, all those different things. So if you want to find me, I'm not that hard to find.

I will put those links under the show notes as well. But dude, thank you so much for coming on the show. I really appreciate this.

Great to chat. Cheers to Shish.

All right. Thanks everyone. See you next time. Thank you for watching or listening to that episode of AI Security Podcast. This was brought to you by techriot.io. If you want to hear or watch more episodes of AI Security, check that out on aisecuritypodcast.com. And in case you are interested in learning more about cloud security, you should check out our sister podcast called Cloud Security Podcast, which is available on cloudsecuritypodcast.tv. Thank you for tuning in, and I'll see you in the next episode. Peace.