Episode 45: AI & Bug Bounties with Casey John Ellis!

You're listening to Across the Pondcast. Cybersecurity rants and war stories from both sides of the Atlantic. Hello and welcome to Across the Pondcast, a show where we talk and occasionally rant about cybersecurity topics with perspectives from both sides of the Atlantic. I'm Tiberius. I'm joined, as always, by my soon-to-be-replaced-by-AI co-host, Mr. Andy Swift. And today, we're joined by Casey John Ellis, founder of Bugcrowd and Disclose.io, and self-described troublemaker and troubleshooter. Casey, welcome to the podcast.

Thank you for having me.

Andy, how are you doing?

You're absolutely right. Greetings.

Well, we're talking about AI.

Yeah. Well, to be honest, I think quite a few of our recent episodes have ended up talking about AI. And I say recent episodes from like the last year and a half.

It's getting a little hard to avoid.

The podcast is only a year and a half years old.

Yes.

It's literally all the episodes.

Yeah, literally everything, whatever we end up going down, whatever topic we pick to talk about, always somehow meanders back to AI recently, which is... Incredibly annoying.

But hey, it's the way of the world.

We're going to talk about it more today. So I'm looking forward to that.

We are.

There you go.

But before we start, this show is made possible thanks to our sponsors. If you want to sponsor an episode, please send an email to sponsors@acrossthepodcast.net for more details. Okay. So before we hit the AI stuff, though, I think we've got Casey John Ellis on the podcast, we need to talk about Bugcrowd, because I know I met, I think I met you for the first time at ShmooCon, like in real life, but I've, um, yeah, but I've like obviously been following you online for a while, but you founded Bugcrowd, which just is insane. Firstly, that you founded Bugcrowd, but also that we have you on the podcast. So talk us, can you just give us like the background of how all that went down?

Oh gosh.

Because I'm really interested.

Well, it started with AI, so there was that.

No, not really.

Look, basically, I, and, you know, mentioned this earlier on, I'm, I'm actually coming off the, uh, the back throat thing, so I don't normally sound quite like this, but if anyone's distracted by that, there's your explanation. Let's just move on. But, uh, yeah, with, with, with Bugcrowd, really what it came down to, like, I grew up hacking in like the late '80s, you know, and the '90s. That was kind of my vintage.

Yeah.

Finished high school, didn't know what I was gonna do. Basically tripped over into a network engineering job and started breaking stuff. In context of that, and then realized that oh okay, like this like offensive you know security mindset, this idea of like enjoying thinking like a criminal but not wanting to be one. There was actual good that I could do with that, and a potential career that I could build from that. That was sort of my experience of that, right? So fast forward to kind of the late 2010s and early 2010s. What I was really looking at, I basically ended up running a pentest company. There's a bunch of stuff in the middle there, but I'll save you guys that part. And, you know, the thing that bugged me about it was the idea that like one person being paid by the hour and being kind of asked to outsmart this like crowd of adversaries with all sorts of different skill sets and motivations and more of an incentive to succeed than kind of constraint based on an hourly rate. Like the math was wrong, right? It's like, it doesn't really matter how smart or dumb that person is. Like the kind of the economics and the resourcing of the setup is just basically doomed to fail on the defender side. And in the meantime, having grown up in the hacker community, it's like, well, you've got this, like, amazing group of people that are sitting kind of waiting for an invite to the table, and they're unplugged from the problem because everyone's scared of them. Like, this seems like a stupid problem for the internet to have. So that was really kind of the core things that were sort of keeping me up at night, that ultimately became Bugcrowd. Like, the big thing about it was I had a series of conversations with customers that I was doing pentest work for. It was around the same time that I I think it was Google and eBay were like starting to talk about their vulnerability reward programs. And I'm like, well, you know, what do you guys think about this? Why aren't you doing it? And, you know, everyone said the same things. It's like, I'm scared of hackers. Like, I don't know how to pay someone in Uzbekistan. You know, my security team's overloaded as it is, blah, blah, blah. And, you know, the light bulb went off. It's like, okay, if we can start to build out things that can actually solve those operational issues, we get to basically shift the economics of how we think about like outsmarting bad guys. But also at the same time, we actually get to start to change the environment for people that are hacking in good faith. Because at that point in time, this is 2012-ish, the idea that like a hacker could be someone who's actually helpful was a completely foreign concept to most people. I still feel like we've got a bit of a way to go in terms of teaching people that these days, but at least it's sort of out of the gate as an idea. And those were really the two things that I kind of wanted to solve. There's a lot of like details that I'm leaving out there, but it's, it's been fun.

Yeah. I was gonna say nowadays it's less like, oh, the hackers are the bad guys. Now it's like, oh, AI, AI can hack. Now AI is the bad guy.

Yeah. Don't get me started on that.

Like, it's just, we've just shifted. We've managed to shift it onto the computers again.

Yay. I'll look at that and think, oh, damn it, we failed.

Does that make us the good guys?

Yeah, I guess so. Like, it could be. Yeah. Look, I think anyone who's worked in, in defense, especially people that have worked in offense for any period of time, you know, they all kind of— like, you guys are in that boat. I assume a lot of people listening to this that are in that same boat. Like, the internet's way more vulnerable than I think most people realize or care to admit, right? The fact that it works still in 2026, I think, is kind of a minor miracle. And, you know, I think collectively the hacker community's helped kind of get it to the point that it's at now, but there's still like a crap ton of work to be done. And what I feel like AI has gone and done is sort of distributed this idea that like, the bomb apocalypse was already here, just not like evenly distributed, right? All of a sudden, everyone's thinking about it all at once, which is kind of where we find ourselves today.

Yeah, exactly. And just going back, because like, yeah, I mean, when you did create Bugcrowd, it is kind of a crazy idea that you're basically— because like, prior to like bug bounty, it was mostly just, yeah, if you wanted to know how secure you were, you would hire a security consultancy, a pentest firm. Bug bounty is more like, oh, we're just going to basically say, yeah, anyone can just try and hack our site as long as you follow a set number of rules and sort of putting it out to trust. The idea on the surface, especially back in 2012, just seems insane, but it works. It works very well. Yeah.

Yeah, I think that's right. Like the insanity part of it I think mostly had to do with the fact that, you know, the preconceived idea of what a hacker is in most people's mind is like a bad thing. Like, wait, no, like we're trying to keep those out, which is why we only work with our trusted pentester, who also, by the way, happens to be a hacker. Like, none of it was rational when you think it through, right? But that's sort of where it was all up to. So like a lot of the early days of Bugcrowd was just education, like teaching people kind of, this is how this actually works. And in reality, like the bad guys aren't waiting for an invite. So, you know, like it's cool that you've, you've got, you know, assurance programs in place. It's cool that you've got people to come in and, and try to break in and try to teach you where risk is and all that stuff. But at the end of the day, I still sort of bring it back to this idea that, you know, our job is to basically try to get ahead. Of the adversary wherever we can and try to reduce risk like that. And they, they've just got a lot more firepower to apply to the problem than we do as defenders. It's like the defender's dilemma kind of personified in a lot of ways. So yeah, it was, it was definitely, there was a lot of eyebrows raised. You know, those early days were really fun because like we got to do some really like provocative shit to try to educate people. And, and, you know, honestly getting the opportunity to work with organizations that even back then Like actually truly did sort of care about this stuff and were willing to take some chances on how to do it better. Like that was, that was the really fun part. But yeah, telling those stories, you know, that there was definitely a joke for a period of time in there that Bugcrowd's actually like just a swag company that dabbles in cybersecurity because there was a lot of like t-shirts and stickers and marketing and just like stuff that we could do to kind of create this like intrigue from all angles on, on the ultimate goal, which is trying to bring hackers and And defenders together to understand each other better, so we can all go forward in a way that's more secure. Yeah, like I said, the memoirs are going to be lit. But it was—it was—but it's been a fun ride.

I have several bug crown swag, so yeah.

Yeah, if you know, you know. Like then it's 100% a thing, and it was—it was pretty deliberate too. I gotta say, like there's a there was a lot of folk on the team, you know, myself included, who loves you know the the marketing and the communication. aspect of this who love like memes and, and all that sort of stuff. Just, you know, for the lulz on their own, right? 'Cause they're just fun. But like they actually really do serve a pretty incredible and powerful purpose. So we pretty deliberately leaned into that quite hard in the early days.

Shows, you know, your community as well, people, right?

Yeah. I mean, that, that is the thing. Like there's, there's so many folk, you know, there, there is definitely this, it works in both directions too, right? 'Cause like for the, for the, the vendor, for the buyer side of things, Like the hump that you've got to get them over is this idea that like, no, these aren't bad people necessarily. Like some of them are, and that's why we're all here in the first place. But there's this group that are good and want to help. And like you're trying to create intrigue to the point where they'll actually initiate a conversation. I think on the other side, on the hacker side of things, a big chunk of it, especially in the early days, but it's still true today, is just feeling, you know, kind of almost vilified. It's like, okay, we're Like the outcasts, we're not really understood. Like we're trying to do all this stuff. We don't know how to, how to even form community necessarily. You know, why don't they understand us? All that kind of stuff. Like creating kind of these, these rally points where people can kind of form a tribe around it or to help do that. Yeah. I mean, like we don't take credit, I think, for, for actually doing that. I think the community was already on the way to doing it itself, but we sort of saw our role early on as, as Making that as easy and as fast as possible.

Enabling.

Yeah, nice. Yeah, exactly.

Yeah. So AI. So let's, so let's talk about how robots are going to ruin all of that.

Yeah.

Yeah. I mean, when we initially, when we initially came, came to the idea of being on the podcast, we were like, I think, can we, can we talk about sort of how AI is changing both bug bounties and sort of pen testing. And I guess, I guess inclusive of that, just cybersecurity in general, right?

Yeah.

Because I think everyone has opinions, but I think we've seen, uh, as well as I've seen, because I'm not really that involved in bug bounties, but a bunch of my friends are, and they've seen sort of like the issues of bug bounty programs disappearing, the bug bounty payouts being reduced, just be— and it's, and obviously the, the main issue, which is like just Especially the beginning of this year, the amount of bug bounty reports just being completely AI-generated. Yeah, a lot of it slop, for lack of a better term.

Well, it's become the term.

Yeah. I mean, talk us through what you've— yeah, what you've seen like the past. I mean, right now we're recording end of July. So the past, I guess, 7 months plus.

Yeah. And I mean, the answer to this is going to be completely different again in 3 months' time because I think this whole thing is evolving. So quickly. Look, yeah, there's, there's a lot of it to me that feels very familiar. Like when we first started running bounty programs in particular, so like open incentivized programs where it's like, come at me, bro, and we'll pay you if you're successful, right?

Hmm.

You know, a lot of companies were, were pretty overwhelmed by that in the, in the early days, primarily because the amount of, like, the amount of, like, firepower that was being applied to the problem at that point. Like, they didn't— they thought they were fine. And then all of a sudden, they realized, no, you're absolutely not. Here's this long list of things that you need to go off and fix. You know, that was definitely an issue for certain organizations. Like, we had to, we had to learn pretty quickly, like, which kind of organizations are going to be ready to integrate this kind of level of feedback into their vulnerability management programs and their risk management programs and all that kind of stuff. It's not just about finding the bug. Like, you've got to actually Then go off and do something with it, right? And you know, there's still in 2026 like a vastly broad, you know, spectrum of levels of maturity that companies have in their ability to actually deal with. Like, and this is the real information before you start talking about the swap and the span and all that stuff as well, right? So like AI has gone and dialed that up again. What I feel, you know, like I've seen over the past 15 years doing this is that there's been different kind of. Advances in automation, you know, around discovery and around risk, like attack surface management, was a really good one as an example. Like when folks suddenly started realizing, oh shit, it's 11 PM and I don't know where my stuff is on the internet. You know, a bunch of hunters kind of came out and demonstrated that, right? And that was really good. It was really important to do. But like a lot of organizations again ended up in a position where they were getting so much of that feedback, they just didn't know how to The thing that is a little bit different, I think, about AI is that, you know, this concept of like, you must be this tall to ride has dropped at the same time as the ability to discover has kind of gone up, if that makes sense. Like the cost to actually discover a legitimate issue has sort of gone down in a lot of ways. And then this barrier to entry has gone down at the same time. So what you end up with is a lot more people jumping into the mix and trying to help out. That just nets out to a whole lot more things being discovered all at the same time. And yeah, basically everyone kind of simultaneously freaking out about it. So that's on the real bug side of things. On the swap side of things, you know, I can still very vividly remember getting, you know, people that were like Photoshopping fake excess returns back in like 2012. Like just to try to get, try to game the system and try to, you know, get a payout or, or whatever else. And, you know, if you step back from the whole thing for a second, like it makes sense, right? We're creating games for people to break things. So like one of the things they're gonna try to break is the game itself. That's always been to me just a fact of running these sort of things. Like that's, yeah.

Yeah.

Like the internet is a crazy place. You can't tell it what to do. It'll do Kind of what it wants, and it'll do what it wants with whatever tools it has available to it. So kind of what we're seeing is like, if you think about like, there was already this like, call it, you know, 90/10 ratio of spam to legitimate issues, they've both been like multiplied by a factor of 3. And you end up with this like enormous gap between the slop stuff and then trying to figure out like what to do with this increased rate of valid findings as well.

Yeah.

So that's at least the nature of the problem, if that makes sense. And like a lot, and a lot of, a lot of companies have responded to it by just shutting down the program altogether. I think a bunch of them have, have responded by basically downgrading like a, a bounty program where you're incentivizing this to just having a vulnerability disclosure program where if people find things, they've got the option to tell you. I actually think that's probably the most sane You know, kind of reaction to all of this on the defender side if you feel like you need to do that. I'm all for bounty hunters getting paid, but at the same time, it's got to work as well, right? And then this idea of, you know, like reward structures kind of changing, like the simpler issues or the issues that are more easy to find kind of getting devalued. And then, you know, the really smart companies in my mind are the ones that are actually really having to think through like what does risk actually mean to us? How do we, instead of just trying to like tamp down everything else, how do we put a greater incentive on the things that we actually really care about and try to draw people in that direction and kind of distract them from all the other crap?

Yeah. Because like that, that reminds me of like, I've seen recently, like just the sheer number of CVEs being released. And it's, I mean, so I think that there's a general confusion about the purpose of CVEs because it is just, ultimately a way to track bugs. But like, yeah, the whole, the whole point about, well, now we have like 1,000 CVEs to fix, and it's like, well, you don't necessarily. You have 1,000 vulnerabilities. You might not need to fix all of them. You might want to prioritize the order in which you do fix them.

Yeah. Yeah.

Um, so yeah, that, that's a good point, I think.

I think the dirty little secret around that is that that's been true this entire time. It's just people haven't been saying it out loud. You know what I mean? Like, it's, it's one of those things where it's like, no, like we are actually not gonna get around solving all this shit. 'Cause it's like, software is hard and, and we've gone off and created a lot of it. We're now at a posi— in a position where we're creating more of it more rapidly.

Hmm.

And like to, to, to think that like theoretically you could make all of that stuff bug-free, like hypothetically, this is like utopian, you know, AI-enabled future where code can be perfect. Even if that's true, like there's still All of the hump in the road to get from here to there. So this idea of prioritization, bond management, and just literally as an organization being able to say, no, we're not going to get to all of this stuff. We need to actually figure out how to prioritize what is most important and then burn down things in that manner. Like I said, I feel like that's been a conversation that's bubbled up since maybe 2018 or so. Like it definitely ramped up during COVID when everyone just went crazy and put all this stuff on. I was like, But now with AI, it's like, oh, okay, we can't actually avoid the fact that this is true anymore. So let's just talk about what we need to solve.

Do you find many companies have like fairly mature programs when it comes to vulnerability management and things like that? Because whenever I talk to people, like even in the pen test and red teaming world, they have like shocking approaches to remediating like the most basic of issues.

No, I mean, honestly, the answer is no. Like, I feel like there are There are a small number of companies, like companies that are kind of post-2008, so like they're cloud-native by default. They've got, you know, they've got like in their development deployment processes, they've got these things that make fixing things easier, kind of inherently baked into how they do engineering. I think they're in a much better position to be mature around this. But then you've got companies that are post-2008 that haven't built that stuff in. You've got, Yeah, the remaining like corpus of organizations that are literally like powering the freaking internet at this point that still aren't that great at that. Yeah, for the better part, like I feel like it's a universal, it's a universal struggle. And I mean, even the companies that do do it well finding it difficult just with this increase in everything. Like it's, you know, everything everywhere all at once. And part of what they're doing is relying on this kind of OODA loop that they've kind of grown up with that's now so small that they can't fit inside it. So they're having to think differently about how they approach things. I think pretty much everyone's having a version of that problem right now.

Have you found, I mean, so obviously I don't get to see many of the internals of what companies are doing, and maybe you don't either, but it strikes me that if you do, if you're a company that has a bug a bug bounty program, you might be tempted to use AI to scan your actual code and do code review more and find bugs before the bug hunters do. Do you— have you found like any increase in that? And that's like helping?

Definitely. And to be really clear on that, with, with the Defender add-on, like, I think companies absolutely should. Like, like, like the goal— this, this can be a really difficult thing to explain to bug hunters. Sometimes it's like, no, like, what you guys are doing, like, you finding a bug and getting paid is not the point of the exercise. The point of the exercise is to assist to make, you know, the software and the system more resilient.

Yeah.

And you're, you're, you're, and you're, and you're, yeah. And you're a player in that story. So, like, the whole idea of, of a company suddenly getting better at, at, you know, how they, how they make their code more resilient before it hits production, like, that's, Kind of the goal here. Like life is meant to get harder for us if we're doing this right. So yeah, it's it's a it's a hard thing to get across to folks sometimes because it's yeah, I just want my payment for my thing. But yeah, if you zoom back out to it, it's like well this is always going to evolve and and this whole idea of like using these tools to be able to actually do security in a in a more yeah reliable way you know in order to protect the user. Ultimately, like that's a good thing. Yeah, there's definitely a lot of that. I do think the thing that you know the the advantage in terms of what you know input from the outside world tends to have is impact on an organization. It hits different when someone internal finds a vulnerability. Like you've got some kid from halfway across the world who's gone and completely owned your staff. Like all of a sudden, you believe in the fact that that's possible, probably more quickly than you know if you. Resident pentester, does it? Yeah.

I, I love that, um, sort of, uh, trying to change the opinions of bounty hunters, just hackers in general, I suppose, to get them to sort of zoom the fuck out. Because I, I, I remember, I remember my first, the first thing I hacked on the first pen test I ever did, and it was the most stupid thing ever. It wasn't anything glorious. It wasn't anything incredible. It was a default Telnet password on a switch. Boring as hell, right? Barely even can call that hacking, but the feeling of euphoria once you got onto it, it was like, oh my God, I really like this. I'm coming back for more.

Yeah.

And that's, that same feeling has cropped up like throughout my career about various different things you do. You discover a bug, you discover a vulnerability, you pop a shell on something. It's that same feeling over and over again. But if that starts diminishing at some point, sort of law of diminishing returns almost, Yeah, I wonder if that affects people and its popularity over time.

I think, I think it definitely does. Like, that's something that, you know, we've taken on both in the Bugcrowd context and in the Disclose.io context is this idea that, like, hackers, you know, like, our job is to— not even our job, like, our wiring, but then what becomes useful as a product of that is to take the assumptions that are out there and basically just flip them upside down. And see what falls out, right? And you know, I think for a lot of us, we're sort of born that way, and then it becomes a question of like how do we actually like cultivate that and put it to work? I think for others, they kind of want to learn it and come into it. It's definitely more difficult now than than it was before to to get into a position where you can sort of stand out and and build a career around this stuff. And I've definitely seen more buck hunters kind of burning out just just because it's like oh. You know, people don't take my reports seriously. Like, am I, you know, imposter syndrome kicks in or whatever else it might be on their side. It's not a new thing at all, but I do think that like the prevalence of that has kind of gone up a lot with AI kind of dropping itself into the knowledge kind of workforce in a lot of ways. And I think that's true in a lot of other fields as well, but like we've seen it a lot here because we kind of like being special, right? Like, We're like the undocumented access feature enthusiasts that know the dark arts and all that other stuff. And when that starts getting more difficult for us, that can get kind of discouraging. So yeah, something that I've thought a lot about is the whole idea of like, how do you continue to cultivate that pipeline and get people interested in things that can give 'em that dopamine hit, like encourage them to learn more, like get them engaged.

Yeah. It is something I've, I've struggled with, um, in my own team as well, is how do you keep them inspired? How do you keep them looking for bugs and finding that hit when it's so easy just to AI the crap out of everything these days? But I think the answer is also AI the crap out of everything because it's, it's opened up this world where not everyone in my pentesting team that I have is, is a reverse engineer or has that capability or that, that mindset yet.

Yeah.

They might be working towards it, but AI can absolutely help you on that journey to understanding how to break down something, discover something, how to start on your journey with reverse engineering. It can give you a massive leg up to get you to the goal quicker, but with still a decent amount of understanding if you use it right and you don't just let it all do it for you. But there are ways to sort of turbocharge and keep that interest going to people that would perhaps have sat back previously and gone, oh, do you know what? That's going to take me like a year just to learn the basics of that. Yeah. I'll probably just bench it and carry on pentesting for now. So for me, it's kind of, it does as much damage as it does good at the moment. I think it's a, it's a really weird one.

Yeah. I mean, I mean, so did the Steam Engine, right? Like, yeah. When you, when you, when you, when you think about it through that lens, you know, I think every, every kind of transformational, you know, advance in technology has that quality to it. And, you know, one of the reasons, I mean, you guys talking at the front of the show about how like AI's crept into everything. Like when I first saw generative AI kind of, you know, not so much in the wild, it was sort of starting to work with Frontier Labs prior to the release of ChatGPT. I'm looking at it, I'm like, okay, this is like the, this is like the internet in like 1997. Do you know what I mean? Like we're, we're about to go through this enormous hype cycle and like a whole bunch of BS and a whole bunch of like trying to figure out the wheat from the chaff and all of that sort of thing.

Mm-hmm.

But in reality, what this will do is go on to basically at least touch pretty much everything. It's just a question of how that rolls out, what that looks like, and, and kind of what kind of timelines are involved and all that kind of stuff. I think, you know, probably the worst thing that a security practitioner can do right now is to is to sort of almost be like a bit of a Luddite with all of it and and ignore it in that sense. I think using it for learning for for newcomers is really is really powerful. I also think that like the more I mean in my experience, the folks that are like really strong in their particular like. niche of security, whatever it might be, are the ones that end up being the most proficient in actually using AI to do their job because they've got the subject matter expertise to actually steer stuff and drive it. Like, they're not delegating their brain out to it. They're using it to kind of amplify the things that they already know. And that includes, you know, I do a lot of stuff with like startups, right? One of my favorite things about all of this is seeing a whole bunch of people that have really good, like, they've got such a deep understanding of the problem. that they're trying to solve at the coalface, right? But they're not a builder or they're not an entrepreneur or they're like, they don't, they've not gone off and built things around that because they just don't have the ability to do that or they don't see themselves as that kind of person. And now all of a sudden they've got the ability to whip together tools that at the very least kind of communicate the concept, right? Like that, all of that stuff to me is super exciting and messy and all of that other stuff. But like, you know, those are the kinds of things that I think people can kind of get into at this point in time if they're starting to feel a bit bummed Yeah, get that feeling of euphoria back.

I was talking to someone that used to work for me about a year and a half, 2 years ago, I think. And they've moved to another firm elsewhere, but he's now creating tools and things like reporting tools for himself over at his new company, like pentest reporting tools. He would've never had that drive and incentive and skillset to do without things like AI really sort of Giving him a leg up to do it. And credit to him, he's used it and he's used it really well at the other end of it. So use the tools and get that feeling of euphoria back.

Yeah. And you don't have to go through the rigmarole of getting permission to work with someone from the engineering team and the brother, like all of these different things that are just kind of, have sort of in a lot of ways kind of gone away as barriers to just doing stuff. Yeah. I want to be careful not to like oversell that because it's not like a silver bullet for everything, but, you know, for folks that are kind of tiptoeing around this or trying to like almost deny the fact that it is a shift that we're seeing in technology. Like those are the kinds of things that you can actually do to get involved in taking advantage of that. And I think they're kind of fun.

Yeah. I've written, well, I say written, I've got AI to write. I know people will crucify me if I say I've written something.

You've combined with your clanker in order to, yes, go on.

I've gotten AI to write so many things in the last 7 months that I use on an almost daily basis. And it's just insane. And they're ideas I've had for years and I just haven't found the time to do it.

Yeah.

And it's insane that just how quick you can get them done. And even, it's an interesting thing because obviously, There are— vibecoding isn't perfect. There are definitely vulnerabilities. Like, people shouldn't just vibecode stuff and put it into production. But Tanya Yanka said something the other day. She hacks purple, for anyone. We should have her on the podcast. She's awesome. Yeah, she is. She said something the other day, which is like, there needs to be sort of— we need to differentiate between sort of vibecoded slop, again, for lack of a better word, and just something that you You vibe code, but it doesn't touch production data. You just run it locally. It just does a singular job.

Yeah.

You don't need, like, it, there's, the risk is lower if you are not putting it in a place where it can access dangerous stuff. And I think that's a, it was a really good point.

Yeah. No, like the house, the houseplants tweet, right?

The houseplants, she called it, yeah, houseplant apps.

Yeah.

Like just stuff that you, you just run for yourself. And you're not like, you're not putting it as a service on the internet for anyone to run. It's just like, here's something that works for me, it works for this particular job, it automates some part of my job or my life that previously was not automated, and it just helps. And I think that's important to recognize as well.

Yeah, no, I totally agree with that. I mean, I, you know, like with, you know, Disclose.io as an example, like something that I have been wanting to get off the leash for a really long time is the idea of basically an arbitrary lookup tool so that if someone's on Internet Safari, they find a broken thing, they can figure out who owns that thing and who they should report that vulnerability to. And to always give them, you know, at least the backstop answer to that question, but then try to get them as close to the right answer as possible. Like, this is an idea that's been sitting in the back of my brain for Forever, right? And I'm like, oh cool, I could just like go build this now. But at the same time, yeah, I know how to build things to break other things, but I'm not a platform engineer by any stretch of the imagination. So like, I work with a guy named Damien Radford, who is another former Bugcrowder, and like, he's, you know, doing the sanity and the engineering and the architectural passes on that stuff. So like, I ideate and then he picks it up and turns it into something that actually is production ready.

Mm-hmm.

At that point in time. And like, to me, like those sorts of, if you're thinking through something as like potentially a platform that gets used into the future, then that's a neat kind of way to do that. Like you actually partner up with proper engineers who understand, you know, maintainability, like security, like all of those different things from like a proper engineering standpoint. Like I, again, I know how to break shit, but don't necessarily have a deeper understanding of how to build securely. He's, that's the thing he's the expert in. So like, you just, there's still like, people still matter, I guess is really kind of the, the point I'm trying to make there. I do think the other thing with that, like Dan Meesler, who's a, who's a really good friend of mine, he's got this, this sort of statement that he throws out, like no robots in the gym, right? This idea that like, and I love that because it, yeah, that's good. Because you, you, you end up in a position where it's like, oh, I've gotta produce a bunch of content. Like, I'm just gonna outsource that to my, to my clanker. But then all of a sudden you find writing more difficult because that's actually the thing that you are uniquely good at. And you gotta find some sort of way to like, yeah, reach a midpoint in that. So it's like, okay, you don't go to the gym to avoid resistance. You go to the gym for the resistance. But at the same time, if you happen to work on a factory floor and your job is to move things from A to B and you're like rewarded based on how many things you can move to A to, from A to B, like that's the perfect place for the robot to come in and help out. Like, just for everyone to be able to think through things in, in that way. A lot of it does come back to this idea of like, what are you uniquely good at? And, and what do you want to get better at? And how does this become like supporting infrastructure for that rather than a replacement to it?

Do you think that, I mean, so, so I think we, we mentioned it, like there's, with AI generating code and AI getting better at generating code and AI getting better at finding vulnerabilities by auditing code, Yeah. There is, there is at some point in the future, gonna, it's gonna come to a point where AI is generating code at such a good standard, or AI is analyzing code at such a good standard that it finds bugs before they're pushed to production. Do you think that where bug bounty hunting and pen testing, like the future of that is going to just shift entirely to sort of testing AI itself? Because the one main thing that LLMs just can't get away from is the prompt injection techniques and those kind of vulnerabilities. And I think there's been so many papers written like, this is an unsolvable problem. I mean, everything's an unsolvable problem until someone solves it. But if we assume it's an unsolvable problem, do you see That we'll just get to a point where most of bug bounty and probably most of pen testing is just testing the agents behind like all these perfectly secure apps?

Yes, yes and no. Like, like I sat in a room where Sam Altman said that a little while back and it's like, oh, okay, that's plot twist. Yeah. Like when you, when you think through what, what an LLM is in particular, like it's, it's probabilistic by So the idea of, of, you know, it having deterministic rules built into it, it, you, you're fighting the fundamental architecture of the thing that you're trying to secure at that point. And there's always going to be limits to that. Like anyone who's worked in security for any period of time knows that that's a thing that works until it doesn't, right?

Mm-hmm.

I think LLMs are no exception. So there's definitely that in terms of like attacking AI systems, you know, trying to like exploit You know, their output, you know, training data injection. There's this entire class of like new attacks and new kind of risks that are out there that we haven't really scratched the paint off of, you know, too much yet. So like that, to me, there's a lot of runway in that for folks that are thinking about how to apply research and kind of what we do in the future. I do think on the other side of it, like not everything gets solved with a pull request, and I think everyone keeps forgetting that, right? Like the idea that Yeah. Okay. We can like audit source code, we can fix open source libraries. We can like get to a point where we're automatically patching all those different things. That's great. Now we've gotta go off and deploy and regression test that patch into 9,000 products that are out there running in the wild. Then we gotta get those patches actually implemented. Oh, and by the way, like that's the top 5 turtles in like a 30-year-old stack of turtles that's actually powering the internet right now. So, Yeah, I don't think that— I think there's going to be a lot of progress in how AI can be applied to solving those things. And I think there are a lot of people, myself included, who are pretty focused on what that actually looks like. I don't think it's going to be an overnight thing in that sense. Probably the last point I'd make there is that even if theoretically we get all of this right, the bad guys aren't going to just pack up and go home. So, you know, like, it's like, I don't believe that we can like magically solve cybercrime by making everything perfectly secure according to what we think that means today. So, you know, to me, the question becomes at that point, like, what do they come up with creatively that we haven't really thought of yet? And in comes the hacker back into that conversation to try to preempt that, right? So yeah, those are, those are, I think, I tried to give like the very practical and also the like the 300,000-foot, like 10 years out answer to the question. 'Cause it's really, it's a really broad thing to think about right now. But yeah, yeah, I think not giving up and, and actually, you know, thinking through like, how does this play out? Like, how do I individually as a hacker, you know, contribute to this stuff? Like, what are the things that interest me that are going to hold my attention as this evolves? That's, that's on its own a really important, I think, really positive thing for people to be, you know, thinking through.

I think there's always a temptation when you think about like cybersecurity for a lot of people just to think about it as in like plain web apps or just simple infrastructure terms. Like the world of security is freaking huge and there are so many areas of technology that even AI struggles to completely have an understanding of to the point where it can break and hack something.

Yep.

It is, it's far more proficient at doing the basics right now. It's very good at it. But, um, when you start to think about like the really niche areas of cybersecurity and technology, there's still space in there for researchers. There's still space in there to discover new things. And, um, you still got half of the industrial world like running on fricking PLCs made in like 2010 or something. And like, there's still gonna be things to find, still gonna be systems to test. And While there's humans still in the loop on the defensive side, making errors and leaving things open and misconfiguring stuff, there's always going to be space for it, uh, for the middle term, I feel. But I mean, but who knows? We'll soon see. What I'd really like is something like, you have like a firewall interface. What I would really like, and my love for Fortinet is well known, but I would love something on like their, I'd love something on like their sort of main GUI page, which is like, Basically Clippy, but for Fortinet is what I would like, where it just comes along and just goes, hey, you've, you've exposed all your admin affairs. Are you sure that's a good idea? Um, that would be a cool thing to have and might solve a lot of problems, but until we have Clippy for Fortinet, people are still going to screw up. So yeah.

Well, yeah. And you raise a good point too, because that's like, I do think that, you know, like a lot of the Like a lot of the things that AI is being applied for from like a defense and a fix standpoint right now are very like atomic, right? Like we've got this vulnerability in this library, blah, or we've got this class of bug in this type of system, blah. And it doesn't necessarily consider the fact that like a lot of the really bad stuff, when you think about taking a bunch of primitives and actually creating an exploit chain from it, it's the relationships between those things, right? So like this is this whole, this whole idea, and that's at an app level, that's at a network level, that's at a, like a system level architecture. You know, that it's more of a graph than it is like a list of primitives, which is kind of how we're treating things right now. So this idea of like, at some point in the future, someone coming out with a way for AI to be able to actually holistically assess architecture and infrastructure choices and like, what do your access control lists look like? All that kind of stuff. Like, we haven't even started doing that yet because we're still so focused on code, which is where we should probably be focused, right? But it kind of suggests this idea that like, there's, there's a long tail to this. It's not going to be fixed overnight.

Yeah. That big picture kind of mentality is an interesting point. Yeah. It's very much focused on like that code level. Yeah, you're right. But then like, when you think about the complexities of most sort of global organizations' internal networks, there is a lot going on there, like architecturally, to get your head around, even for an AI to work out all of those little pieces of infrastructure, how they fit together. Like, just, just designing Active Directory on its own is a nightmare in some organizations.

Yeah.

But when you consider all of the other bits and bobs they've bolted into their networks, like there's companies we work for that have been going for like 20, 30 years, just plugging any old crap into their network and acquiring other companies that bring along their crap as well, and trying to understand and piece that together in a sort of global context and work out where their actual vulnerabilities actually lie. Yeah. That's a, that's a really hard problem to solve, but that's, yeah, that's, that's not where we're at now and that's not where we're going.

And in, and in, and in, in the meantime, everyone's using this stuff to make like stupid decisions faster with more energy. Right. So like, like you could, you could argue, I think pretty convincingly that we're actually adding to the problem in this particular season that we're in right now.

This particular season. I like that.

And we're not gonna necessarily realize that until like shit starts hitting the fan a little bit later on. So like, yeah.

Yeah. Maybe season 5 and 6. Yeah. There's gonna be, yeah. Just, just you wait.

Yeah. I don't know. Like, like to me it's, it's, it's always a funny one 'cause I'm like, I'm clearly massively conflicted in, in saying this, but that's for a reason, right? Like, like I, I do think that like human creativity and crime and exploitation of this stuff is ultimately a product of human incentive. And human creativity is the thing that ultimately kind of makes that work, right? So on the other side of things, you've got, okay, I'm going to build this thing. I'm going to use all the tools that I've got available to do that. Like I've got an incentive to make it work. I've got to, you know, reduce the incentive to make it secure because that's not actually the most important thing to me. The most important thing is it working in the first place. I think you put all of that together and, and, you know, automation, I think in all of its forms, including AI, can kind of reduce the surface, but you're still going to have these primitives. In the mix. Like the idea that humans are the ones that are building stuff for other people to use. And then you've got humans that have the incentive and the motivation to attack that stuff for whatever outcome they want. It's just the kind of the playing field that we're doing all of this on is sort of shifting at the moment. But if you take that out of the equation for a second, I think those primitives still remain true. Even if you, like I said before, even if we theoretically solve Quote unquote, if there is such a thing, every problem that we think exists in security today. That doesn't mean that there aren't going to be new ones tomorrow. And there, it doesn't mean that there aren't going to be new kind of attacks or new motivations on the bad guy side for us to be thinking about in the future. That to me is a product of like, that predates the internet as a concept, right? Like that, this is not an AI thing or even like an internet thing. It's just how it works. Yeah. Sorry, getting very, very philosophical here, but hopefully it's like— no, I think about this stuff in terms of how it maps down to the practice of what we're talking about, right?

Yeah, it's getting very philosophical. I need to bring it back to lobster rolls just to— yeah, just to get a brief respite.

We didn't record the lobster roll bit. Nobody will know the reference. No, only we will know the reference to lobster rolls.

There you go.

I'm just gonna leave it in.

Yeah, yeah, just leave it in anyway.

You see Swift at DEF CON.

Yeah. If you see me at DEF CON, ask me about lobster rolls. Um, I won't know. I won't be able to give you any answers until like after Friday.

This episode will come out after DEF CON, so that's—

Yeah. Unlucky.

Yeah. Um, damn. So I know, I know one thing you wanted to talk about, Casey, was vulnerability research, uh, specifically with AI. And I know like AI is like, when AI first came out, I think, I don't think anyone thought it could, it could even get to that stage. And we've seen recently. It solved the Jacobian conjecture in math. I read, I think today it's like they launched Mythos against some crypto systems and it found vulnerabilities in various crypto systems. So where do you see vulnerability research specifically going?

Yeah, I think, you know, it's going to be, because I mean, in the middle of that, you've got all of the like export control freakout stuff that's going on in the US. You've got You know, the kind of OpenWeight model advances happening out of China and other parts of the world. There is definitely a, like an international relations overlay to all of this now that's very across the pond in that sense. Putting that part aside, yeah, I think the thing of it is that like there has always been a lot of bugs in code. And I think the thing that people are realizing at this point in time is that that's That's a real thing. Like, that's a true statement. Yeah. I've seen a chunk of that throughout my career through like the application of the crowd to this type of thing, doing that at scale. Like, you just see a lot of stuff fall out and realize, oh wow, there's a lot to find. But now we're starting to concentrate those efforts and you're starting to see like streams of issues come out of different places and whatnot. It's an interesting one because I do think that, yeah, we're going to need to start to think about like how we fix this stuff in a hurry. Because, you know, the more people, like the more sophisticated these models get and the more people that have access to them, again, like the lower the cost of discovery's going to be. And at that point it becomes a question of what are they going to do with that, right? So like we're pretty rapidly heading into that season at the moment. And yeah, I think it's, you know, to me, like a lot of the advice that I give to researchers at the moment is to try to figure out how to apply your focus to areas that other people aren't necessarily thinking about. So, you know, like everyone's looking at open source software, like, you know, code, like all those different things. That's, that's great. That's awesome. Like, what about hardware? You know, what about cryptography, cryptosystems? Like, how does, how does AI's ability to adapt attacks against like post-quantum cryptography, like what does that even look like? You know, just this like hypothetical streams of research for people to think about. It's it's really kind of going back to something, Andy, that you said earlier around like bug bounty and and volume research being often thought about as like just a web thing or just a network thing. Like no, zoom back out for a second and think about okay, how does all of this plug together? Like what are we missing? To me, like some of the most valuable things to be thinking about right now would be like okay, what is being missed? Yeah, what is that? What are what are the frontier labs and and even Like the commercial application of AI and, and vulnerability research, you know, gonna take a while to get around to that's still relevant from a risk standpoint. And how can we kind of get the jump on that? Because there's a long list, I think. But yeah, in the middle of that, like I think about vulnerability disclosure, you know, this whole kind of increase in volume and the way that it's like causing the cybersecurity industry to actually explain what the hell it's doing, which I actually think is a good thing, but we weren't necessarily ready for that.

That's interesting.

Like, wait, why are we, why are we paying you for all this stuff when a client can do it? Like, it's, it's, it's not the That there isn't an answer to that question. It's just that I don't think that we've all been asked that at the same time before. So now it's like, oh, like everyone's kind of freaking out. But yeah, in the middle of all of that, like organizations just kind of deciding to put their head in the sand and say, hey, I don't want to listen to the outside world because it's too annoying. That is a very real thing that's sort of going on right now. And yeah, to me, like the, the risk of that is the idea of like all of the work that we've done to establish The hacker is like a good faith component of the security ecosystem. Yeah, the baby is definitely at risk of getting thrown out with the bathwater on that one. So that's that's a concern that I have and something that we that we work on a lot with discloseio because it's like this is not you know if you build stuff like no matter what you do to try to make it secure, you're going to have missed something because that's just how this works. And someone outside your organization will eventually find that and then they'll try to tell you. That when that happens, you've got to be ready for that. And that's just literally a function of the physics of the internet. So you don't get to opt out of that. Yeah. Yeah. Trying to make sure that, that, that continues on as, as a thing that's in people's minds is a primitive of how all of this works. Right. And that's not like a Bugcrowd thing or a 1 or a Synack or any bounty platform thing. To me, that's just, you know, something that people need to be thinking about and ready for. So yeah, those are some of the things that I have, like, in, you know, swirling around in my head on that. I do think offensive security Like in terms of capability development is going to really ramp up because in the middle of all of this capability stuff, you've got, as I said before, like a world that's getting spicier and there's, you know, people that are doing more things with more capability and there's got to be responses to that. So I think that's the thing that's coming over the hill that's worth thinking about as well. But yeah, there's a lot of, you know, there's a lot of things in the mix right now, that's for sure.

It's an interesting time to be in our industry. I think it's I don't know if I'd want— if I zoomed forward like 10, 15 years' time, I don't know if I'd want to be starting. I feel like we've had the best of it, like, in the start, and then we're, we're now living through the turbocharged part of it, which is just getting everyone excited.

The 2010s were awesome. Like, the 2010s were fantastic, honestly. Like, that was, that was such a fun period to be doing this because it was like you had a receptive audience and you had all of this like ground to educate in and cover and all that kind of stuff. And it was just, it was Yeah, to me, there was kind of an ease to it. That is definitely, you know, the advantage, I guess, right now is that cybersecurity is back to being like kind of a hot topic. So it's easy to have a conversation about this stuff, but then figuring out what you actually do with how you try to drive people forward from a, from an action or direction standpoint, that's getting noisier and more difficult. I think.

I have a big question for you now. Good. Are you ready for it?

Lobster is good.

Loves it. I knew that was coming. Excellent. Set myself up for that. Yeah. Big question, right? Big question. Then or now? 2010 or 2026? For what? Pen testing. Let's go pen testing first. We can do a few of these.

Um, yeah. So I, I think now, honestly, because I think we've actually got a better chance at Delivering outcome that matters, that makes makes an impact to the customer and to their users from like a risk standpoint. I think for a lot of a lot of the time that I've been in the game, you know, pentesting has been a thing that you sell to people that don't care, right? And that's when you end up with like you know vulnerabilities that still exist four years later in the same test. I think that's becoming less of an option at this point in time. So like the like the accountability behind it and and the fact that like this is an open conversation that yeah, you do suck sometimes and you got to fix it. That to me is, is, you know, when I'm doing security testing, like, I'd like to see the thing get fixed. Do you know what I mean? Like, there's definitely, you know, an element of the industry that sort of doesn't care. It just sells the thing because that's the thing that people are buying. And that's fine. It's valid. But I do think that that's coming under a lot of pressure. And at the same time, there's this increased sense of like, no, what are we actually doing here? What can we actually fix?

Hmm.

Yeah, that's, that's a long-winded question, but like, to me, Now is good because there's a lot of opportunity to just make everything better, frankly. And I think that's a good thing.

Yeah, I think there's a lot more reception to our words of wisdom that come out of our reports these days than there ever was before. I think times gone by, we were perhaps seen as a bit of a sort of dark art. And there's always a sort of element for clients receiving our reports of, oh well, that probably won't happen to us. We'll put that on the back burner. And then several years later, you find the same issues as you said. I think that's— you ask any pen tester, I think that is probably going to be a frustration of seeing the same reports regurgitated over and over again in history.

100%. But like, people, people do believe that the boogeyman's real, like in a different way. Yeah. In 2026. Yeah.

Yeah. Yeah. There's a lot of— yeah. The difference I think is that, yeah, boogeyman was, was real, but the chances of bumping into him obviously were quite remote. But now, now, yeah, yeah. Now you've got, now you've got AI at the front of it spewing out boogeymen all over the places. And the chances if you get getting hit by it are that much more elevated. So maybe it's done us a favor in a way, in terms of just making the rest of the world sit up and going, crap, we need to actually fix this stuff that these guys have been banging on about for the last 4 years.

Oh, that's what they were talking about. Okay. Got it. Right.

Yeah. Got it now. Now it's a problem. Yeah. That may be a good time. I do look back in the old days with sort of rose-tinted glasses, just, I think because We were young, just entering the industry and like, you never forget your first few sort of pen tests, do you? Um, always, always good fun, right? And some of the places like we went with Tibbs when we were working together, like we had a lot of fun, like going on site and doing like random shit wherever we ended up and it was, and that was good. So yeah, I, I think roasted into glass is part of it for me. Yeah. I'm, I'm hopeful for the world we live in now, it's more receptive.

Yeah, no, I, I agree. Like, I, I mean, on that as well, like I do, I've got a, a story arc that I use in, in presentations a fair bit, like One of the funny things about Bugcrowd is that when I first landed in the US from Australia, it was the same month that Snowden did his thing. And like, when I look back, yeah, when I look back, you know, one of the positive kind of impacts of that is, you know, to me was one of the first times that like the entire internet thought about security in the cyber realm as something that's potentially relevant to them all at the same time. Like that hadn't to me really happened before.

Yeah.

Right. So, you know, all of a sudden you've got this like conversation that you can have and the ball gets rolling and, you know, you can start to actually do stuff. That's the upside of it. On the downside of it, it's like, oh man, I kind of miss just doing this for fun.

Yeah. Right.

I think that's, you know, it's a perfectly fine thing to say. I don't think it's like not fun now, but like the idea of like, no, this is commerce now. Like this is a real business. We have real relevance and a real opinion. We kind of always wanted that, but it is like easy to be kind of like the dog that caught the car. Shit. Yeah. Well, well, it's all kind of nice too. Yeah. Yep.

I like the idea of a dog catching the car as well, by the way. That rings so true. Like my dog annoys the hell out of Tibbs because he sits in his bed down here and he snores throughout most podcasts that are in the day. He's lucky today. He's in the other room, but he's a little pug. But if he ever caught anything from the sort of pigeon size up He'd be screwed. He didn't know what to do. He's also been going mental about it. Yeah. Like, what would he do with it? Like, he chases pigeons and squirrels. He's never going to catch one, but if he did, what would he do with it?

I just want to go on record for a minute and say the only reason I don't like your dog is because the snoring, the snoring makes editing a nightmare. So, all right. All right. That's the only reason, Swift. That's the only reason.

So you gotta just, you gotta, you gotta put together a comp track where it's just the dog snoring for a little while and you can kind of play it out.

Honestly, I reckon we could make a whole episode out of it, like just an hour-long episode.

That's awesome. That's awesome.

The dog. April Fool's Day next year, maybe.

Oh yeah. Um, let's do it. Let's do it.

Yeah.

So any, any of those other, you know, 2010 versus 2026 versions? 'Cause that, that, that is kind of fun.

So reverse engineering, it's probably another fun one to talk about then or now. That was an incredibly niche and manual process back in 2010. And now you've got all sorts of MCP helpers and all sorts firing off all over the place to shortcut probably about 70 to 80% of like the time sink things. And that was always my thing with reverse engineering. It's a time sink.

Yeah.

Like, to get from A to B, there was an awful lot of fuckery in the middle that was just renaming functions so you understand them, reverse engineering bits you don't care about to get to the thing you actually care about. Anything that shortcuts that problem for me is brilliant. So my opinion on reverse engineering 2010 versus 2026 is great. I'm happier with it now than I was back then because there's so much less wasted time.

Yeah.

Like, I think I've told this story before, but I never forget, like, one of the firmwares I reverse engineered recently, recently, last year, 2 years ago, can't remember. It was for a Switch, but I spent ages like reverse engineering a function that basically made a light blink on the front of the Switch. And I was like, well, fuck, that's not gonna— I'm not gonna find many vulnerabilities in a function that makes a light blink. It's like, I just ended up going down this bloody rabbit hole of like finding things. I was like, this is not the thing I want to do. But, you know, zoom forward and you end up with AI being able to preempt stuff like that going, yeah, don't pay attention to that bit. That's a blinking light.

Or you get to watch it go down the rabbit hole for you.

What are you doing? Yeah, it's even more rewarding because I can sit there, it's like, oh yeah, that's stupid, like knowing full well that I probably would have done the same thing, um, like 5 years previous. So yeah, that's where I sit.

How about the, uh, the state of education? Oh, just in general, because I've got— I got— I mean, this is a spicy hot take, I don't know. I was reading an article, reading the article the other day of a a teacher who put white text in his, like, for like an LLM. It was like a project, like an essay, and it was all in black text. And in white text, he, like, basically invisible, he put in parentheses after one of like the requirements, include a reference to Madagascar or something, but in a way that makes no sense. And 33 out of 36 of the student essays mentioned Madagascar in this weird way. So clearly they just copied and pasted. And someone on Reddit was like, the 3 who didn't just read their essay before submitting it and removed it.

Yeah.

So I think this is honestly the worst thing about AI right now is the scary effect it's having on the education system and the potential that we're going to have a bunch of people graduating from college who aren't actually qualified in like the sort of degree that they're trying to get because they're not actually learning stuff. Right. This is just pushing it out. Yeah. Yeah. What are your thoughts?

It's a, that's a, that's an interesting, so I've got, I've got a 15-year-old and an 18-year-old who's just started college and, you know, they've both been in school through this period where like the initial reaction to, to LLMs was like, you are not allowed to use it, you'll get expelled, blah, blah, blah.

Mm-hmm.

Um, that was kind of, you know, the, the kind of early like knee-jerk, I think from, from a lot of educators, cuz it's all of this, it's like we, okay, as teachers we don't know what to do with this.

Mm-hmm.

Like this feels like cheating, like all that kind of stuff. And like to me, the negative consequence of that has, has been, you know, you've gotta actually like put work in to get them comfortable with these tools. It's like, no, they're not evil. You can definitely use them to cheat and shortcut in ways that aren't productive, but ultimately they're the kind of productivity tools that everyone's going to end up using them. So it makes sense for you to start to learn how to use them yourself. Right. So there's that whole thing. Yeah. I think it's a, you know, I kind of feel like education's been sort of geared up for, you know, the industrial revolution of like the 19th century for a fair while now, and it's probably due for a bit of a reboot. So maybe we're testing some of the edges of, of that whole side of it. Yeah. Like a lot of it to me does come down to like, how does this stuff actually fit into work, into, you know, what we do in the future? Like, I, I, I don't see it being absent from, from any of that. You know, go to the dentist and they take an x-ray and it shows what the problem is. And then the doctor comes in and like verifies that. But AI's the thing that actually gets them to that answer more quickly. Like that to me doesn't feel like cheating. That just feels like better dental care. So I don't know, it's like, it's going to be a weird— I don't think that one's going to be evenly distributed at all. And what I do come back to is this idea that like, if you just ignore it and try to make it go away, it's like you'll warrant it.

And so that's a big topic, I think, the education. Yeah, it really is. And I think we've— we're very privileged because we do— we live in this golden era of a moment where we have the experience of the before and the now. We've got something that can We can sort of steer and do a lot of stuff for us, but it's that ability to steer because we have network engineers, programmers, developers. So wait, they're the same thing. Hackers, pen testers, et cetera. But they're all— wait, they're the same thing too. But we have all these—

Red teamers.

It's 20 past 10 over here. It's my bedtime. But we have all these things. We've lived through all these professions and we can We can sort of steer the tools in the right direction at this point in time. My worry is in the future, are we going to end up with these sort of black holes, these voids of people that are just essentially prompt engineers who don't know of their background? But at the same time, there's all this years and years of higher education don't just go away. And those structures of teaching people don't just evaporate overnight. I think they change.

Yeah.

I don't think everyone's just going to completely de-skill over the next 10 years. And we're just going to end up in like that scene from From WALL-E, this film from years ago. I don't think that's going to happen. I think it's just going to shift how we educate and how we learn a little bit more than anything.

Yeah. Yeah. I do think it's a good thing to be ahead of though, because it is one of those ones where it's like, all right, to me, the defining— you can't teach an LLM wisdom. You can teach it knowledge, you can teach it reasoning, but you can't necessarily teach it wisdom. And like wisdom is the thing that you learn by screwing up, right? For the better part or getting taught from other people's mistakes. So like just little things like that where it's like, okay, what's the role of, you know, what does education look like into the future? And then, you know, frankly, something I did a keynote on this at BSides Las Vegas 2 years ago.

Yeah.

Because like I actually, in the middle of all of the, you know, Buckeye stuff and all those different things, I actually had to have Heart surgery, which was not expected, but it was, yeah. And it worked out well. Thank the Lord in the context of all of that stuff. But it gave me a lot of time to think about like, what does this legacy look like? What does like this kind of cross-generational handoff look like? All those different things. Because ultimately, you know, none of us are kind of around forever and like the younger generations coming in are the ones that are going to inherit this problem. So how do we Like make sure that we're sort of passing on the wisdom bits that we've learned either through education or hard knocks or whatever else to them. You know, in my head, like that looks like it's working when they're able to take advantage of these tools, but then like not skin their knees in the same ways that we did, if that makes sense. It's an interesting one to think about because that whole thing, you know, I think a whole lot of people are freaking out in that space all at once. And, you know, the freakout's good if it prompts, you know, a question of like, what do we do next? Right?

Yeah, this podcast has got so philosophical. It's great.

It's pretty deep, isn't it? I'm sorry. It's pretty deep.

No, no, it's good. Yeah, it's good. We forced our listeners to think about philosophical subjects and seafood and seafood and seafood lobster rolls without any context, which is great.

Is it though? Is it really?

Seafood in 2010 or 2026?

That's a good question right there. No, look, I think we're going to be good. I think the mindset and the hacker spirit, you know, to me, like, it's all the deep tech, like, go down the rabbit hole and nerd out on the different aspects of it. Like, I love all that stuff, but if you zoom back out and think about what we are, you know, our role is to break shit, like, take assumptions and tip them upside down and see what falls out and try to leave things better. You know, as a result of that. And like, to me, technology doesn't ever really replace that. It just gives us different things to play with and kind of changes the playing field that we're doing it on. So as long as we're sticking with the important things, I think everyone's going to be fine.

There's no one better at breaking our own shit other than humans. We're very good at doing that.

We're very good at that. I love the question. It's like, is AI going to cause the downfall of humanity? It's like, no, we're doing a pretty great job of that on our own.

Yeah. We've got to hold my beer. Yeah. Yeah, we got this.

Well, Casey, thank you so much for joining us. Please tell our listeners where to find and follow you online.

Yeah, absolutely. So I, I write stuff at cje.io. The Disclose.io project is at disclose.io. Bugcrowd's at bugcrowd.com. I think, you know, if you want to get in touch with me, I'm on all of the socials, but yeah, the, the, the blog is kind of the main place that, you know, is the, the central point of connection and communication. For people that are looking to, you know, dig deeper into, into vulnerability disclosure, how to do that well on, on the defender side and on the researcher side, definitely check out Disclose.io because we've got a lot of tooling that we've been working on there. It's just growing up. We want your feedback. We want your help. I think making sure that, you know, this, this whole ecosystem of hackers stays healthy and in the fight as all this stuff evolves, that's something that we're all very interested in making sure we see happen. So, you know, whoever wants to join that fight, feel free to reach out.

Nice. We'll put those links in the episode description.

Thank you.

Thank you all for listening. I've been Tiberius, he's been Andy Swift, we've been joined by Casey John Ellis, and this has been Across the Pondcast. You can find all of our social media links in the episode description or on acrossthepondcast.net, and we will see you next time. Farewell.

With lobster.