Bug bounties in the age of AI

Hello again, Internet. It's your pal, Todd. We have been here all day across the street from Moscone Center here in San Francisco, California, USA, planet Earth. And we have been talking to so many cyber— the hoy cyber hoy, maybe?

The thought ops, the thought lords. The thought lords.

And here we have another another one. His name is Casey Ellis. We have been old friends for quite a while now.

Has been a while, yeah.

And we're here to talk about bug bounties in the age of AI. Yes. I like that. That's cool. And so hello, Casey. Thanks for coming by.

Hey, thanks for having me, Todd.

It's cool. So I think on the website you are listed as Casey Ellis, founder of Bugcrowd. This is still true.

That will always be true. I wear a great many hats, a number of hats these days. Yes. Best known as the founder of Bugcrowd.

But that's not what you do today.

No, no. I mean, obviously—

It was a point in time.

As you said, once the founder, always the founder in that sense. But I actually stepped back from the full-time operating role in Bugcrowd at the end of January in this year.

Okay.

So, you know, letting Dave and the crew kind of take the ship and run with it, which is cool. But that's giving me, you know, obviously a bit of time to smell the roses and touch grass and do all that kind of stuff. But, you know, there's ideas that have been cooking away in the background that I want to get working on more aggressively. And that's a part of what I've been doing.

Was it, was your step back because you're terrified of AI?

No, no, honestly, actually, that's a really good question because I am getting asked that a lot. Yeah, I think, yeah, we started working with OpenAI back in like '22, just before like ChatGPT dropped. I've been, you know, like as security people are wont to do, fiercely critical of the hype and the snake oil side of things, but recognizing pretty early on that like this is tech that has transformative implications to it. So to me, it's really a matter of like, you know, how do we think about all of these things together? To me, it still starts with the security researcher because when you think about the bad guy, they're human, they've got intent, they're going to end up with the same tools as us. So there's always going to be this like spy versus spy element to it. But, you know, the bits in the middle are in the process of shifting. So, yeah.

So I, just as an aside, I have spent a lifetime, you know, playing Sid Meier's Civilization.

Yeah, yeah, yeah.

And when you play Sid Meier's Civilization, the name of the game is tech advancement. And the name of the game is like, I want to show up with battleships against your triremes, right?

Yep.

But that's not how our world works. Like, no, we have roughly, I would say, and maybe exactly the same technical capability on offense and defense. Is that accurate?

I— we have the ability to access the same technical capability on both sides. I personally think that offense is just inherently more agile than defense. You know, unless you're working for a nation state inside, you know, a large government office building. Sometimes that's a little slower, but everyone else is pretty fast. So there's a— there's an agility and a nimbleness that offense has that defense sometimes struggles with, but otherwise I agree with that. Right. Yeah.

And so, and so I guess I bring it up because like we're all getting AI-ified at the same time.

Yeah, 100%. Yeah.

Is there— so let— so, so all things being equal, it's that offensive agility, is that, is that still the discriminator or does, you know, does LLM assist on offense or defense? Is there an inherent offensiveness or defensiveness with LLM security?

Yeah, I think that it's a really good question. I think, you know, where we end up—

This is the beauty of not discussing what we're talking about.

Yeah, I love it. I think, you know, I've always believed that like defense is offense's child, right? Like you don't, you don't go to the trouble of like a hole between— hole in the wall between two rooms will serve adequately as a door, right, if the goal is to go from A to B, right? But then it's not secure, so you do all the other things. You don't do those things, go to the, the expense and the inconvenience unless you realize there's a, there's a reason to do that, and that's usually something offensive happening that kind of educates, you know, that need. So I think as a primitive, like, you know, defense is kind of the child of offense. And, you know, AI lets you move fast. I mean, on both sides. I think the thing that we've seen a lot and something that's been kind of keeping me up at night, but I'm trying to lean into and facilitate as well, is that like we're actually really bad at writing secure software.

Yeah.

And we're really bad at patching things that should have been patched a long time ago. Like the internet's a lot more vulnerable than I think people in security get this, especially if they're on like the VR or the VM side of things. But even outside of that, like, I think people are a little shocked at how bad things are. Yeah. So, you know, what we're— where we're at now is that we've got the, like, you must be this tall to ride bar has gone down a ton, right? Which means there's more people that can jump in and do kind of whatever the heck they feel like, good and bad. And the things that they're capable of achieving, you know, through the combination of the, the advancement of the models, but also like just how much fragile opportunity there is to do stuff out there on the internet. I think that's kind of the bit that we're in the middle of right now, or just the start of right now. And I think that hump is going to be pretty bumpy, like getting to a place where we're actually cleaning some of these issues out get to the point where defense does actually serve, AI does serve the defender more effectively than it does someone doing offense. I think we can arrive there, but it's going to be a bit of a bumpy road to get there. Would you say, so like, I mean, how spicy do you want me to get, man? I can do that.

We can get hella spicy. Oh hey, my boss left the room. We can get very spicy.

There you go.

Boom, done. Your boss is still here though. Defenders, right? Like, they, they operate, you know, at the, at the behest of offenders, I guess. Offensive. Um, defenders, I think— and tell me, tell me if you agree or disagree, or tell me why I'm wrong, right? Sure. I think defenders in this space are naturally going to be a little more reluctant to change their tooling, a little more reluctant to adopt like anything labeled agentic AI because it's scary. Yeah, sure. And people on offense are more likely to adopt it faster because the penalty for failure for them is like, eh, I crashed my target.

Right, right. Oops, oopsie doodle, not my target. They're pursuing a yes/no, whereas the defender's pursuing everything.

Okay. What does that mean?

Well, I mean, you can run harder at offense because your goal is just to get across the line. Right. Like as the defender, you're trying to stop all of the ways that that might happen from happening. It is like the defender dilemma at like AI speed-wise, right? Is a pretty interesting thing to kind of think about.

It's like when you have— so I heard this joke from Stop Podcasting Yourself, which is a delightful Canadian comedy podcast. Right.

Guy was—

and it made me think of this because he was like, I don't understand why some exit doors are labeled warning, you know, this door will— this door is alarm. It's like, what do I care? I'm leaving. This sounds like a you problem.

It's usually not true either.

And I feel like— and I've been thinking of that for like 2 days now because I've just heard this, right? And I'm like, Oh, okay. So if you're on offense, your penalty for the worst case— the worst case in an action, right? If I have farmed everything out to my AI agents, right? Is that likely worst case is the attack fails.

Yeah, you don't get shell.

And you get no shells. It's like, oh darn. And then you can go back and say, why didn't I get my shell? And I can iterate on that.

That. Yep.

Um, I guess there's the case of like, oops, I got caught. Like, yeah, how much is that?

I think that's become— well, this is— I mean, all of this stuff's happened at the same time because I completely agree. I think we're about to go with that is as a defender, oops, it dropped the production database and now my company can't make money, which is like a different part of the structure.

Defense game now.

Yeah, I mean, on, on the tactics side of things, like, we saw this, um in 2000s during COVID like basically, you know, the pandas switching from, from COVID to just brute. Yeah, like let's just minigun the entire internet, right, and get whatever we can from a shell standpoint and set it up opportunistically for later. Um, that's continued, and I think other actors have jumped in with a similar approach. Um, you know, basically just ignoring stealth.

Right.

Like this whole idea of don't get caught, like that's a Western cyber espionage and cyber warfare construct. It's not so much the kind of thing that we're defending against. Yeah. So to me, like if you look at what's gestures vaguely—

Yes.

All around the place. Yeah. I kind of wonder how much longer that's going to be important for.

Yeah.

In terms of like operations. Yeah. At which point if you got all these tooling, sets available to you, like, why not just join in the whole kind of minigun the internet thing? Theoretically. I'm not advocating for that.

Sure. Sure. Theoretically. In the game.

Yes.

In the game, you mean. So, OK, well, with that downer— well, that's the thing. Like, is it likely the internet is going to be more dangerous, or is this a recency bias of the present where the internet has always been barely secure enough for most people most of the time.

I think that, like, I think that's the part that everyone's starting to realize. It's like, holy shit, like, how is this thing still on, right? Um, and I can tell you which, like, it's the nerds. Yeah, it's the nerds. I think it's the nerds. I think there's partly— I don't want to invoke the whole kind of idea of mutually assured destruction, but like, it's partly like, if this goes down for one, it goes down for all, you know, that kind of thing. There's some there's some rules of the game that seem to have held up okay on top of the valiant work of all the nerds, right?

We had norms at one point, right? And those norms kind of evaporated around 2020 for reasons.

Yeah. Yeah. So, you know, I don't know. I think I did a talk, I mean, on the recency bias thing. I do think this whole kind of idea of like the geopolitical, like, who is a threat actor, who is a good guy, who is a bad guy. And like that's getting blurrier and blurrier and there's an acceleration to that in terms of how you think about offense and defense on the internet specifically, right? Throw AI on top of that, throw accessibility to AI to a broader array of threat actors on top of that. It's like you got yourself a stew at that point. And I do think that, you know, I did a panel in DC 2 years ago basically You know, the question I asked, this is, I think, a good thing to think about. Like, if we've developed cyber defense doctrine and kind of the rules of politeness on the internet through, you know, 10 to 15 years of peace and prosperity, right? And we're now transitioning out of that. Um, and this is before you sprinkle AI on top of all of it, right? Like, what do we need to throw out, revisit, you know, double down on? Like, what are the things that we've traditionally seen as like not okay that might become okay? And, you know, what do we need to be thinking about that we might not already be thinking about? I think that as a thought experiment is actually, is a really good time to do that just because of all of the, you know, things that are kind of shifting right now.

So I don't think people are gonna like just become more evil because it's easier to be evil, right? I think there is still a place for bug bounties.

Yep.

Right? Like, there are people who like finding bugs.

None of this has anything to do with Bug Bounty.

Right. I'm trying to slowly steer back in. And so that's the thing, right? Like, in this, I guess maybe when I was thinking of the title, it was like, oh, Bug Bounties in the Age of AI. Well, AI is destroying everything. So, you know, are bug bounties even applicable? And I do think they are, right? Yeah. Like, with— so bug bounties as an outlet paying for the labor of people who like solving these puzzles, right? Who don't want to do crime. They have no interest in doing crime. And they don't want to hurt people.

Yeah, they want to, you know, they want to contribute to safety and defense, not like offense and harm.

And incidentally, you know, not mind getting paid.

Yeah, exactly right. Yeah, so that still exists. It does. And I think That piece where I think we're at at this point in time is the bug bounty. Because I mean, you know, you and I were jamming on this stuff before I started Bugcrowd. Yeah, the primitive to it is vulnerability research and the fact that people on the outside of an organization find vulns. And people inside the organization write vulns because people aren't perfect. QED. Like that's the end, right? There's no real debate on that one. There's also this idea that if you add an incentive, you can actually encourage people to do that and then try to direct their behavior towards the things that are most important for you to learn. And this is where bounty, you go back to the Wild West kind of definition of bounty where it's you just, you use—

Or the Star Wars definition.

Yes, Star Wars. There's a million of them. But the idea that you're using an incentive to reduce information asymmetry between a disparate group of sources and a single source. Like, that's what a bounty is and does, right? Um, I still think that that's— I think that's perpetually relevant because every bug is worth something, right? Like, it costs something to put there, it costs something to find, it costs something to fix, and it has value to someone who wants to exploit it, right? That value ranges from zero, which is where all the slop stuff's happening right now, which is why everyone's freaking out. Okay, right. All the way up to, you know, you hear about handset O'Day going into the intelligence community and things like that in the million-dollar range. Like, to me, I see it, what we're talking about, actually on that spectrum. And I think that continues to exist. Where I see it get shifted around a bit is where it's been confused with like pen test or just assurance where like people aren't actually fixing things anyway. It's just kind of a You know, the places where we've sold security to folk that don't actually care, I think those are going to come under a lot of pressure as an industry just in general over the next period.

What do you mean by that? The places where we're solving security where people don't care?

Selling security to people.

Oh, selling security. Yeah.

So like the whole idea of like, have this pen test. Well, yeah, cool. The only reason I'm doing this is because the compliance guy told me to. I must check the box.

So I'm going to find the cheapest thing that's called a pen test.

Exactly. So what's the value of that? Checking the box. It's got nothing to do with risk reduction necessarily. So if that's true, it's going to have a lot of pressure on it and kind of go away. Right. And hopefully what's left behind is this idea that like bugs are worth something. You know, how do we think about that and actually plug that into how we think about risk going forward? That's what I'd love to see happen.

Well, speaking of bugs being worth something, What's going on with Disclose.io?

Disclose.io!

Disclose.io, your one-stop shop for things and stuff.

We've been busy. Yeah, exactly right. So Disclose.io started off as really an effort to standardize, like the mission of it is to make volume disclosure suck less across the internet. Like how do we just make this standardized, normalized, get everyone on board. Kind of view it as this primitive of how the internet works instead of this like optional bug bounty or pentest light thing. Um, and a big part of the goal was to try to get the, uh, the DOJ to amend the Computer Fraud and Abuse Act.

Sure.

Which, you know, eventually we got the charging rules changed in, in 2022, right? Um, partly that was a team effort from all sorts of different places, but you know, what this project contributed to that boilerplate language that got adopted.

Yes. Um, lawyers, by the way, this is a major—

I got copy-paste stuff.

But what I'm saying, like, like, if you ask a lawyer, it's like, oh, can I just like copy-paste this thing? Can I boilerplate this into my terms of service? They'll say no, absolutely not.

You need a lawyer.

You need to pay a lawyer to write this thing over and over and over again.

Yep, yep.

So to get a bunch of lawyers to sign off on the notion of boilerplate is an amazing thing.

Yeah, no, I appreciate it. It's been, it's been a journey. I mean, I've been working on that since like 2014. We created Disclosure as a brand in 2018. So that's kind of the key thing. What we've just done is added basically a programs database that goes through every vulnerability disclosure policy that we can find and actually classifies it.

Okay.

So it pulls out the different attributes. So if you're a security researcher and you're looking for where to submit a thing, or if you're looking for where to hunt, you can use that as a basically a vendor agnostic guide.

Okay.

For, you know, what good might look like. The other thing that we're wanting to do with that is to really kind of establish this idea of a maturity model.

Okay.

For organizations. Like, the top of the top is you've got full Safe Harbor, you've got proactive CBD. Yeah. Like, you're making yourself accountable to researchers as an organization, and you're also making them feel safe as they do that. There's very few organizations that actually do that, but like, that's like the top of the hill, right?

And then metric in there of is a CVE numbering authority?

No, not yet. That's a good idea.

Is it a good idea?

We should jam. No, I mean, look, honestly, I think—

because I don't know, right? Like, I've said to people in the past, it's like, oh, well, like, you can trust that this tech company, right? Like, if you're going to give— if you're doing research and you find a vulnerability and then you go down this disclosure process and you see that they're a CNA.

Yep.

Oh, they probably know what they're doing.

They're probably doing something right. Yeah, yeah, exactly. Yeah, okay. So that's for sure. Like, I think of those as like proxy indicators of security maturity in general. Yeah, yeah. But also for the researcher, you can kind of expect better, you know, a better experience off the back of that. So there's that. We've also got an open source database of threats made against security researchers. Like Attrition.org style.

Oh, okay.

Which is kind of fun.

So like cease and desist, I assume?

Yep.

Hey, you're a good person to ask. How many people have gone to real court for disclosing a bug? Like, because I know a lot of people worry about the cease and desist. And for a normie who's not married to an attorney, like getting a cease and desist can be really scary.

100%. Yeah.

Um, but then like people will ask me, it's like, oh well, I got the cease and desist, what'd I do? I'm like, um, well, you either cease and desist or don't, because all this is is a letter from somebody with a fancy letterhead. Like, this is not from a court.

Yeah.

This is not— this is not we're about to arrest you.

Yeah, correct.

So I'm curious, do you know off the top of your head how many people have been lately or ever—

I can't answer that off the top of my head.

I think it might be like than 10.

It's, it's low. It is low. I think if that's the point you're trying to get across, like the actual usage, it may be as low as— I know that that's true. Um, I don't know the exact number. Yeah. Um, and that's talking about CFAA. Like one of the, um, one of the stories in there was, uh, some researchers in Malta that hacked a train. Yes. Um, good faith, like disclosed all that stuff, got their door kicked in. Yeah. And we actually came in through the— yeah, we got arrested for that. Um, we actually came in through the Security Research Legal Defense Fund, funded their legal defense and got them a presidential pardon. And now there's like legislation being written in Malta to carve out good faith hacking. So like, tell me more about— it still happens, right?

Uh, yeah, it does still happen. Yeah. And like, it happens in Texas, it happens in Missouri. Like, our friend Harley Geiger is always on the, the, uh, the drumbeat of like, cool, we did something cool with CFAA, we gotta do it in all states. Yeah, 100%. Tell me about the Security Research Legal Defense Fund because I just heard, uh, the story about these hackers in Malta.

Yep. Yeah, yeah, it's a nice little segue there, right?

Yeah, but, but like, it's not just for US or—

correct—

or usual five, it is worldwide.

Yeah, 100%. Um, so yeah, the SRLDF, Security Research Legal Defense Fund, uh, Jen Ellis and I joined the board there, uh, about 3 weeks ago.

Oh wow, at the same time? Yeah, holding hands.

It's an Ellis takeover or whatever you want to call it. It's good to be working with my honorary cuz. But yeah, the idea there, it's a 501(c)(3), so people donate into it and what we do is accept grant requests from people that have had legal threats against good faith security research. And then we'll basically write them a grant to fund legal representation. So it's, you know, like the, the big use case that pops up is like kids that go on internet safari and end up stumbling into a dark corner and they don't know what to do on the legal side. They might not have the means, right, to do anything about it, right? Um, so yeah, it's cool. Like, I think that—

and this is distinct from EFF's action, right?

Correct. Yeah.

Okay.

Yeah. So, so like the whole thing there is like Okay, you're maybe in Malta or maybe in Australia, maybe in India, like wherever you might be, not in a jurisdiction where American lawyers are going to be super helpful, for example. That's a big thing as well. So like we'll help you find that and actually fund it.

So like the far off land of Canada.

Yeah, Canada is a great example. Exactly. Yeah, so that's probably the big thing with that that I've been doing this week is trying to make sure that researchers that that's available to them and that they can reach out. And also, you know, we're obviously always looking for donors and people that want to come in and support that type of project financially as well. Sure.

Yeah. Well, that's great. I love keeping kids out of jail.

I think it's a good plan.

Is there a pipeline from the Security Research Legal Defense Fund to Disclose.io to gainful employment?

I mean, I think you can kind of get a picture of the ecosystem. And that, you know, I'm traversing here. Like, I see all of these things as related because, you know, security researchers, going back to where we started, I believe are going to continue to be the tip of the spear in terms of how we think about securing the internet. That'll forever be true. So we've got to protect that ecosystem and make sure it functions. And even, you know, with what we're saying with AI coming in and kind of tipping everything upside down right now.

Right.

that's all the more reason to lean in on that type of thing, I think. So, yeah.

All right. Well, I think that is a great place to leave it. Thank you for not being so doom and gloom like all my other guests today. This has been great. Like, look it up. Security Researcher Legal Defense Fund and Disclose.io, both very open source, you know, kumbaya. And we're going to take a break for about 5 minutes, and then we're going to be back with HD Moore, Jared, and Zakir, and we're going to be talking all about all of those weird things as part of the decibel land of this whole situation here. So we'll be right back, 5 minutes, catch ya.