AGI-Pilled Cyber Defense: Automating Digital Forensics w/ Asymmetric Security Founder Alexis Carlier

Hello and welcome back to The Cognitive Revolution.

Before getting started today, I want to take a moment to introduce our newest sponsor, Granola. You probably know Granola as a leading AI note-taker, but it's in fact much more than that. Because it works at the operating system level, it can capture all audio in and out of your computer, allowing it to take notes not just on meetings, every podcast you listen to, every video you watch, and if you choose, everything you say. Right now, to help new users make the most of the platform, Granola is featuring AI recipes from entrepreneurial thought leaders, including several past guests of this show. There's a Replit recipe that converts discussion notes to a Replit app build brief, a Ben Tossell recipe that creates content production plans, and a Dan Shipper recipe that looks across multiple sessions to build an unspoken company culture handbook. My own recipe, which you can try now on Granola, is a blind spot finder. It looks back at recent conversations and attempts to identify things that I am totally missing. This has already proven useful in the context of contingency planning for my son's cancer treatment, though I am very happy to report that he is still doing extremely well. And over time, I expect it will become invaluable for suggesting AI topic areas that I've neglected and really ought to explore. For today, my guest is Alexei Carlier. founder and CEO of Asymmetric Security, which just recently came out of stealth. In response to the launch announcement, Logan Graham, who leads the red team at Anthropic, described Alexei as one of the most AGI-pilled founders in the space. So naturally, I had to find out what that means for the future of cybersecurity. As you'll hear, Alexei's motivation is increasingly familiar but nevertheless profound. If we assume that AGI is coming and that it represents a near-infinite supply of intelligent labor, The question becomes, how should we redesign our cyber defenses from the ground up? His answer is to move from a paradigm of reactive emergency triage to one of proactive, continuous digital forensics. We begin the conversation by describing the current threat landscape, distinguishing between the spray-and-pray tactics of financially motivated criminals, the more sophisticated ransomware attacks of cybercrime gangs, and the patient, high-stakes IP theft operations conducted by nation-states like China. Alexei also shares fascinating details on the North Korean remote worker phenomenon, where state-backed actors infiltrate Western tech companies not just to steal secrets, but also to earn salaries that actually fund the regime. From there, we turn to how Asymmetric is building AI agents capable of performing the deep investigative work that was previously only available from a very limited number of expensive human experts. We discussed the jagged frontier of current model capabilities in the security domain, and why, though off-the-shelf models can already achieve 90% accuracy on many investigative tasks, Asymmetric is going to market with help from insurance companies with a services-first business model focused on business email compromises, both to ensure that they deliver consistently for customers and to build the proprietary dataset needed to close that final gap in performance and reliability. Importantly, Alexei also makes the argument that while most AI technology is inherently dual-use, the fact that people who specialize in these investigations don't tend to become particularly outstanding hackers suggests that digital forensics could be a domain that allows us to differentially accelerate defensive AI capabilities. And arguably, it's this strategy of intentionally shaping the AI capability frontier that is Alexei's most important contribution. And I would love to see this approach ported to other domains. The opportunity to build specialized datasets, evaluation methods, and training environments that can help harden society's defenses seems both increasingly tractable and urgent. So if you're thinking this way and building, whether in cybersecurity, biosecurity, mental health, or anything else where a DAAC strategy could make a big difference, please reach out and tell me about it. With that, I hope you enjoyed this overview of the cybersecurity landscape and preview of the future of automated digital forensics with Alexei Carlier of Asymmetric Security.

Alexei Carlier, founder at Asymmetric Security. Welcome to the Cognitive Revolution.

Thanks for having me.

So yeah, I'm excited about this conversation. An interesting little endorsement of you and your point of view that I saw online on Twitter from Logan Graham at Anthropic, who leads the red team efforts there, and really kind of is at the intersection of national security concerns and what can models do, and has been deep down the rabbit hole of how weird the world might get in the not too distant future. He said about you, in response to the announcement of the new company, that few people are more AGI-pilled than you. So I wanted to start off by just getting a little bit of your worldview. You can tell maybe a little bit of your background too, and kind of how you came to be one of the most AGI-filled people around. But How did that happen? And even more so, what does it mean to be so AGI-filled?

Yeah, I saw this. I suspect what he's getting at is I've taken throughout my career professionally, done a lot of stuff that's been really premised on the idea that AI and AGI is going to be a big deal. So a few years ago, I was on the early team of the Centre for the Governance of AI, which is a big AI governance think tank, especially prominent in the UK and US and elsewhere. And this was like in the pre-ChatGPT, I don't know, 2020, 2021 days when AI people were talking about it, but it wasn't really this big thing it is today. And back then we were trying to think about what would happen as we saw that it was going to be a bigger and bigger deal. And there were just very few people doing that at the time. And today, sort of continuing in that with asymmetric security, where we have quite a distinctive thing we're doing in the space of cyber defense. And I think because we're taking AGI very seriously, like, one frame on what we're doing is we're assuming something like AGI will exist in the world and then be like, okay, what makes most sense from a cyber defense perspective assuming that? And that takes you to quite a different place than lots of other security companies. And if we're wrong, the company's maybe not— it's still valuable. It's way less valuable than it would otherwise be. And so it's like these big costly bets on, like, yeah, this is a thing that's happening.

And what does AGI mean to you? Because we've had multiple rounds now. Like, Tyler Cowen at the O3 launch said, okay, I'm calling it, this is AGI. Friend of the show Dean Ball called it with Claude code with Opus. And I have to say, it's certainly getting there, right? Where do you think we are on the Is AGI here or not? If not, what is missing in your perspective that you think is kind of going to tip us into a stranger future?

Yeah, at least how I think about AGI, it's not here yet. I have in mind more the Dropit remote worker style version of things where you're really fully substituting for remote work that humans do, including for long horizon types of tasks, and you can really fully sub in for humans. At the moment, we have this this weird jagged frontier of capabilities where the models are geniuses at some things and then just cannot string together tasks over long enough horizons. And I think that sort of explains why we have this weird paradox of on some dimensions, clearly they're superhuman, but then they clearly are not having that much economic output just yet. And so I think for me, we will see it in the economic output and GDP statistics and so forth when we really have this, like, you're actually subbing it for human work.

I find the GDP question so confusing because so many of the most impressive moments that I've experienced with AI have been GDP destroying, because it's when it's effectively substituted for something that I would have otherwise had to go out and hire done that I'm like, this is where I'm really feeling the AGI. And so I have kind of a strange confusion, I guess, around Like, more output, sure. You know, I would expect more services rendered, right? More legal services, more, you know, more medical second opinions, obviously lots more code, but potentially at a lot smaller prices than previously. So that's kind of a weird— I don't know if you have a point of view on that, but I still kind of find myself just like, I don't know, the frontier model developers' revenue is exploding.

Yeah.

But we don't quite see it in the GDP statistics.

But I'm not even sure what I would expect to see.

In some ways, I might expect to see GDP shrink.

That's interesting. Yeah, I suspect probably the more relevant thing to be tracking actually is just output. And there, I think very clearly we should expect a lot more than we're seeing today.

Yeah, obviously becomes a little tricky. How do you measure output? Because dollars are the standard way. And it's, yeah, gonna be tricky. I guess just because I've been so deep down this rally, like, like so many, I don't know where your, um, Claude code or open code or, or, uh, MultBot journey is at the moment, but I have been, you know, like many, trying to, um, figure out how I can create leverage for myself. I've done over time like tons of task automation type things where I break down a task and make a little eval set and build a workflow. But that doesn't feel like AGI. That feels like task automation. Now I'm kind of like, oh, we're getting definitely a little closer with this Claude code setup. Like, it can figure things out in really remarkable diversity of different situations. The one thing I have noted, and I'd be interested in your reflections on this, is, and it feels like it's probably going to be pretty easily fixed, but the one thing I've noticed recently is that the Opus 4.5 in Claude code has a strong tendency to try to write code with heuristics for things that I really just wanted to spend tokens on to understand with its own fluid intelligence. So a random task was backfilling transcripts for the podcast onto the website, because we didn't do that in the early days of the podcast. And I was like, well, it'd be really nice to go back and have transcripts for all those episodes. But so many times in the backfilling process, I see it like write a little Python script with guesses about how things are gonna be, and then that doesn't quite work. There's too many edge cases, whatever. And I'm just like all capping it, use your fluid intelligence, just read the file. You know, if you just read the file, you'll know what to do. Don't, you know, write these kind of roundabout Python scripts. That's, I definitely think that speaks to jaggedness. And it also speaks to, in my mind, like, how subtle and kind of honestly minor some of the major weaknesses still are. Like, it just has this strong impulse to try to code up heuristic-based guesses about how things should work, when it's like, if you just read it, you know, you're definitely smart enough, Claude, to know what to do. But it just has kind of some wrong impulses in a few different ways. I mean, maybe we should save your response to that and contextualise it in what you're actually building. But if you have any immediate reactions, I'd be interested to hear them, because I do. I mean, Logan commenting that you're one of the most AGI-filled people out there, I think that is a strong statement, knowing who he spends his time with. So I am interested in kind of your take on how you understand jaggedness and how soon it might be resolved.

Yeah, I think my take here is roughly that RL is a big part of what's going on. with the pretraining paradigm, you were getting a lot of generality kind of for free because you were reading all the internet and the models were just improving in this very predictable way. And I think a lot of what's happening now, and I think Kapathi and others have said this as much publicly, is because you're sort of relying on verifiable rewards, the areas where they're getting good are really kind of predictably things like coding and math. And So I actually suspect that until we have some big other paradigm, I don't think they're going to get that great at writing poetry, for example, for quite a while. That's generally the high level.

We'll put a pin in that. We'll come back to some of those related questions as we get deeper into the problems you're solving and the way you're solving them. I'd love to take a little survey of cybersecurity broadly. and how AI is beginning to impact that and how you expect it to impact it more and more over time. Again, this is an area that really confuses me, so this is a great learning opportunity. On the one hand, I kind of work from the assumption that everything is hacked all the time, and I've gotten so used to it that I would confess to not my key privileged passwords or passwords I don't reuse that are for key accounts like my Gmail and so on. And then I just have kind of throwaway passwords that I use for like long-tail stuff. I'm like, ah, if anybody, you know, compromises any of these services, I don't— I could probably live with that. And I've just gotten to the point where I like click ignore when Chrome pops up that one of my passwords has been found in a breach. So that's the level of security I'm operating with. And I would have to assume that like, you know, it puts me in pretty good company as being certainly exposed in some ways. And yet life mostly goes on. And like not only does like critical infrastructure continue to run, But even I, with my terrible practices, mostly don't have any problems. So I'm a little bit confused about the landscape today, including who are the attackers? What do they want? Who are they attacking? How are they attacking as kind of a baseline? And then how is AI starting to change all that? I'd love to just get kind of a survey rundown from you.

Sure. Yeah, I think broadly the majority of attacks are by relatively unsophisticated amateurs or criminals. Maybe this is like roughly 80%, something like that. And often these are financially motivated. And then on the other end, you have these like very sophisticated attacks from nation states, like a small minority from a volume perspective, but often they're going after like really critical IP or things like this. And so some of these like low sophistication groups will do things like spray and pray attacks, sending out like phishing emails to as many targets as possible. There'll be script kiddies, so using malicious programs that others, more sophisticated folks, have come up with, but just sending them out en masse and hoping it works out. That's sort of one end. If you were being targeted yourself, that's probably what you would be seeing, probably because you don't have a ton of super sensitive IP or stuff that really sophisticated folks are going to come try to steal. And probably you're just like, you're not going to click the phishing email and you're fine. And then at the other end is the nation states. And here it's a very different ballgame. Like, a few important actors here. You have China, a big one, obviously, the CCP. A big focus of theirs is IP theft from R&D-heavy industries in the West. AI is almost definitely like a huge focus now. Russia is another big player. They're interestingly a bit more focused on political disruption. So famously there was the 2016 US election disruption. And another big one here is actually North Korea. which has very different motivations, actually largely about funding the regime. And a big thing they do here is they have this North Korean remote worker program where they send literal North Korean operatives to just get tech jobs as remote workers in foreign companies just to fund the regime. That's like a pretty unusual one, but happens a lot.

I've heard a little bit about that, but I didn't realize it was like a—

Yeah, it's a thing. We actually just had— we just brought on someone from from CrowdStrike who worked some of these cases where they, the companies in question realized that something was up with their employee. And they were both sending back money and sending back various bits of IP.

So the play is to get the job, get access to either information or some sort of financial—

Exactly.

Tooling that they can then siphon money off.

Exactly.

Get the Brex card, get the ramp card ASAP. Yeah, that's fascinating. Okay, so we got China, we got North Korea, Russia, obviously a player in this as well.

Is there a middle?

There's also kind of what— like, who is it when a hospital— this recently happened, um, in a— to a medical system locally here. I went to see my primary care physician and he was like, ah, it's been a nightmare. You know, our stuff is totally inaccessible, and there's some cyber gang somewhere that is demanding ransom or whatever, and they don't want to pay it. And so I'm like taking notes on paper. Who's that?

Yeah, one of the most common types of attack actually is ransomware. And these are often organised criminals who are moderately sophisticated and act in this coordinated way. In particular, what they tend to do is they find ways of getting access to systems, encrypt it all, and then just put up a notice on the screen being like, hey, if you want your data back, send us a bunch of money. There was a huge pop-off during COVID actually. And this economically is one of the most damaging things. Relatively recently, Jaguar had a big ransomware case. And if I recall correctly, they needed a $2 billion loan from the UK government after this happened to avoid going under. And their operations were down for like a month, 2 months, something like that. So very significant economically. Interestingly, the type of attack they're doing means they operate very differently to nation states, where the motivation of the ransomware folks is to be discovered as soon as possible once they've encrypted the data. The nation state, more sophisticated guys on the other end, are just trying to stay sleuthy for as long as possible. And so you end up with these very different modes of operation.

Yeah. Okay. That's quite interesting. When you talk about like a $2 billion loan, that makes me wonder how much of the, you know, in like business, there's always this question of like, how much consumer surplus are you creating? How much of the value that you're creating are you, you know, as a business able to capture? How much of the damage, I guess, that the cybercriminals are causing, do they manage to capture for themselves? If the damage to Jaguar was X billion dollars, that doesn't mean they paid X billion dollars to the hackers, right? Is it like a 10 to 1 ratio, 100 to 1 ratio? Do you have any sense for how much they're actually able to get from the damage they do?

Yeah, I don't know the specific numbers, but it's incredibly negative sum. I think order of 10 to 1 sounds right. It's just incredibly economically damaging. And I've heard some economists claim that if you sort out this sort of cybercrime, it's actually like a huge effect on growth. Because it's so value destroying.

Hey, we'll continue our interview in a moment after a word from our sponsors. In terms of the baseline situation, of the attacks that are happening, and maybe you could— I don't know if it makes more sense to segment this by volume. Probably not. Maybe more like by damages to the degree that we can. How many of these attacks are of the social engineering variety versus finding purely technical exploits where somebody left, I don't know, a port open or whatever the kind of common things are? And how many of them are like these real cinematic type of things where people are really figuring out exploits that nobody knows about? Obviously, that's got to be relatively rare, but how should we understand how much damage each of those kinds of things is doing?

Yeah, I think a bit hard to say in terms of damages. A bit more clear on volume. So the majority is social engineering or phishing. Hard to say exactly, but say like roughly 70 to 80%. The reason for this is kind of simple. Just attackers choose the path of least resistance, and there's no reason to burn, like, say, a valuable zero-day if you can just get away with sending a convincing email. And so that's what most people do to start with. And in terms of the technical risk, most actually comes from known issues. So it's like vulnerabilities that people are aware of, and just most systems have not been patched. Yeah, just the, like, a large fraction of what happens is just people haven't done the basics, they haven't patched what's known. It's, it's like very solvable stuff. Um, zero days, on the other hand, are a lot more rare, um, and tend to be exploited by the most sophisticated actors like nation states. But you hear a lot more about them than you would expect based on what's like what you're actually seeing in the wild in terms of what attacks are going on all the time.

So what works in terms of Defense? Like, turn on your 2-factor auth. How much does that get you? Update your operating system. How much does that get you? If I do those things, am I good? Or how much risk remains if I do kind of the basic common sense things?

Yeah, it depends a lot on who— like, good against who. For most people who are not the targets of nation-state attacks or aren't big corporations who are going to get ransomware, doing the basics like MFA, regularly updating your software, checking your supply chain of vendors, checking their security, having some sort of automated monitoring systems, doing occasional compromise assessments. So proactively looking into a system to see if it's been compromised, that will get you pretty far against lots of these amateurs and some organized crime as well. Frankly, on the other end, it's just extremely difficult defend against nation states. It requires, like, if you really want to do this in a very determined way, if you've got some IP that's super important, you need things like extremely strong limitations on software and hardware providers. Like, you probably just couldn't use most SaaS, for example. You need to be super careful about who your employees are and vet them in a way that's just super outside the norm and plausibly illegal in some contexts. You need to be way more stringent on proactive security. Like, searching for zero days in your system, doing compromise assessments all the time. It's just very difficult to do, and most organisations are just not there, which is why most nation states just have a lot of ongoing intelligence operations that are stealing IP all the time.

I've been advised I didn't make it this past summer, but I want to go to China and participate in some form of intercivilisational AI safety dialogue and hopefully mutual understanding building. And when I was thinking about going this past summer, people advised me, OK, you can't take your devices. You have to get burner phones. The second you get back, you got to throw that phone away. Or you could maybe sell it on or something. But absolutely don't take your devices. Don't log into anything while you're there. Take all these extraordinary measures. Do you think that's necessary if I am just a regular American? Kind of AI yapper taking a trip to China?

Certainly if you didn't do it, you would probably get people reading over your stuff.

And by that, do you mean, like, they demand my phone at the airport and take a look at it, or that they have a way of getting into an Apple device and—

The latter. Yeah. You should assume if you do that, that people are monitoring your communications in an ongoing way. And maybe that's fine for normal people. Maybe that's often fine. I suppose most people would not love the idea of the CCP reading all your stuff. But yeah, I think people generally write that if you go to China, you should expect stuff like that to happen, especially someone like you with a public profile and so on.

And that would continue. Just to understand the architecture of the surveillance, it would be something they would be putting on the device such that when I take that device home, that comes with me. Not something that, like, because I'm on the local network there, that they would have access to only while I'm there. Is that the correct understanding?

Yeah, you should assume they will have some way of getting a persistence mechanism. This might be a physical thing, or it just might be they've got access to your login credentials or something like this, but assume they'll have persistence.

So that's a little bit like, how should I sort of reconcile the, idea that I'm not that important, for one thing, with the idea that there aren't that many zero days to be burned. Presumably Apple, I think I should trust in terms of, you know, they've got a big team of people that are working on this all the time, right? And identifying these things and closing them down. So how is it that they have so many of these mechanisms that they would use one I can't be that high on their power rankings of targets, right? So how— are there just a lot more zero-days than I have been led to believe? Or like, how do I resolve that, that seeming tension where I just don't feel like I should be worth using these things? Because presumably that gives, that gives Apple some ability to like figure out what's going on and close it down each time they use it, right?

Yeah, Apple security is generally pretty good. I think broadly what's going on here is just, it's extremely difficult to get visibility on what nation states actually have access to. From an offensive security perspective, and the amount of effort in terms of like people going into this is you know estimated I think in the hundreds of thousands for the CCP. So there's just a lot of man hours going into finding zero days and finding ways to break into systems. And often, like there are many cases where access that nation states have had weren't discovered publicly until decades later. And so we should just expect that there's a lot of forms of access that. really sophisticated organizations like these have and are using all the time. And yeah, it is the case that, you know, for most normal people, you won't be high on their priority list. And so they're not likely to spend these, like, very zero days if they expect to be discovered. But certainly it's within the realm of capability. And in many cases, they probably can just get away with stuff without anyone knowing for a very long time.

So how do these things tend to come to light? This starts to get a little bit into what you're doing. So I don't know if it would make sense to talk about how people discover things first, or maybe just talk about how the landscape is changing in light of AI, because I think pretty much everything we've talked about so far has been baseline. Like, you could have said all the same things pre-GPT-4, right? So, yeah, maybe what's changed since GPT-4 class models have come on the scene? How has this landscape started to shift, if at all, in a meaningful way? Again, it doesn't feel like it's changed my life yet, but everybody's telling me it's going to.

Yeah, I think for the most part it hasn't actually had a huge effect just yet. It helps with— obviously you could automate phishing emails and things like this, but there hasn't been this really big upflip from an offensive perspective yet. I think that's about to change, or it sounds like on most measurements that the labs and others are doing, the models are already on the precipice of these offensive security capabilities. And broadly, I think the way to think about the effect here is bringing down the threshold of sophistication necessary for any given kind of attack. So attacks that pre— say, moderately sophisticated ransomwares, assume that a load more people will be able to do these all of a sudden. That's not coming online yet, but will be coming on soon. And from a defensive perspective, I think also hasn't been super transformative yet. And again, I think probably about to change. And that's part of what we're building at Asymmetric.

So how do we measure these capabilities? It's one of these weird things. I mean, I guess this is becoming increasingly common, right, where it's like, Across the board, when the tasks were easy and anybody could tell if the AI was doing a good or bad job, things were pretty easy. We're now in so many domains in a world where very few— because the performance is so good, very few people can even really critique it. I'm staring down the barrel of an interview next week with the founders of Harmonic, who created this Aristotle model system that got the IMO gold alongside, obviously, OpenAI and DeepMind. And it's like, damn, I would score zero on that test. So how do I even understand what's going on? I basically feel the same way in cybersecurity, and it feels like there's a very— it seems like very limited number of data points we can get to measure how good these things are when I know there are some things like, well, there's been some zero days since the training cutoff, so we can see if they can do those. But my sense is those are like pretty small number. So how do we even get a handle on what the model capabilities are in a way that we can trust or base our plans on?

Yeah, I think this is super tricky in cyber for a reason that you were sort of gesturing at. One big thing going on is unlike, say, software engineering, where you have a tonne of code on the internet. Most cybersecurity stuff is just not public. In the context of incident response, for example, when a company gets hacked, they're not going to share the logs from the email getting count publicly. And so it's very tricky to actually get a sense of how exactly the models are performing in these contexts because you've got nothing to benchmark on. And I've talked to some of the folks at Frontier Labs. This is a big bottleneck right now. on trying to evaluate because you just have nothing to benchmark on and you're not seeing all the attacks that are happening out there. So this is a big problem. And secondly, also, yeah, the models are getting sufficiently good that you do need a bunch of subject matter experts who themselves are very good at this. And there just aren't that many such people. Like CrowdStrike, one of the top incident response companies in the world, they have a team of roughly 60 people. It's not huge, who can do this really sort of deep forensic defense cybersecurity investigations. So you're just really constrained on talent and really constrained on data. And it does make it very hard to know what's going on by default. I think there are ways to solve this, but by default, things are tricky.

Hey, we'll continue our interview in a moment after a word from our sponsors. So I appreciate you taking a long time here to just give me the kind of baseline lay of the land. How are you going to make this better for us so we don't have to worry about this? And maybe you could start with a little bit of like, what does it look like today when a company starts to get the inkling that they have been pwned? How does that come to light? What do they do today? And obviously you have a different vision for how that can work in the future. And I think being more thorough, proactive, playing to AI strengths are a big part of that. But take us through the kind of before and after of how things are and how you hope you can change them to be.

Yeah, so the problem we're trying to solve basically is make it much, much easier and much more accurate to detect when you've been breached and have that happen as quickly as possible. Today, roughly speaking, you have these monitoring systems, detection systems that are based on static rules, like basically software that hardcodes, oh, this thing is maybe suspicious. And this kind of works a little bit, like it does do something. But you end up in a situation where you get a lot of false positives, a lot of alerts going all the time. And the reason for this basically is that a lot of activity that could be suspicious is also just could be normal. So like if you're logging in from different places or devices, yeah, that is maybe weird, but also maybe you're just traveling to that this week. That happens. Maybe you're just up late in the middle of the night and that looks suspicious, but it also could be normal. Maybe you're downloading files in bulk again. Sometimes you just download files in bulk. And without ability to reason over the forensic evidence in a lot more detail, you just have no way of distinguishing between these things. Also, by the way, on the other hand, there's this genuinely suspicious behavior that looks normal. So if you were trying to exfiltrate data, you might just, over the course of months, exfiltrate very small amounts which are just not very noticeable. And then that typically detection systems at the moment won't pick up on at all. And again, if you had something like a human who suspected there was something going on here and had the time and energy to reason over all this evidence very deeply, you'd probably be able to figure it out. But that's not how this stuff works right now. And so generally, status quo, if you have some alert go on in, say, an enterprise, the security operations center that handles all the stuff will do triage. So an initial, basically rapid assessment. to figure out when an alert fires fast and quickly, like, is this something worth prioritizing? Is this a real alert? Is it a false positive? Can it be ignored? And then in the minority of cases where they're like, yeah, this is actually a suspicious thing, let's pull out and do digital forensics, which is this methodology of doing very deep security investigations where you're trying to figure out in a lot of depth what exactly happened, looking at every evidence source you need to. to figure out all the details. Intuitively, it's just like a detective, like a forensic detective, and you're sort of reasoning over all the evidence you need in this sort of judgment-based way. So that's the status quo. I think what will happen, you can sort of think in principle, what would be ideal here? We know that digital forensics is the sort of best way of doing this. If you had sufficient time and sufficient energy and sufficient labour to do this, The reason you can't right now is because you're just bottlenecked on people. There are just very few people with this expertise. They're very expensive and they work slowly, and it would just be impossibly expensive to do this. And so at the moment, it's this thing that's mostly used reactively after a breach that I think will not stay the case going forward. One more general intuition here. I think a lot of people, when they think about what to do with AI, like when you're building AI agents, what should you focus on? Have this frame of, oh, this is like a human workflow that's really important in the world. And what if I could just automate that work with AI? And that can work. Obviously, you're automating legal work, you can do stuff like this. But I think a better question is, what would you do if you had 100x the intelligent labor at your disposal? Something like that. And I think the answer is that for many kinds of work, you can perform work that was previously too slow or too expensive. You can perform it at scale now because of the change in economics. And that's the bet we're making in the context of defensive security. This domain that is currently super accurate and super costly, only really applied proactively. You could just imagine eventually at the limit getting this down to the cost of compute, and you just have these sort of close to continuous proactive assessments of systems in a very deep way. And that's what we think will happen eventually. The question is, how quickly can we get there? And can we get there soon enough that we can deal with a lot of the security risks that are going to come on with AGI and offensive security changing and that kind of thing?

Yeah, I like the frame of, okay, you could take, usually for guidance, usually say you can probably save 90% time and money on some task that's being done by humans if you automate it with AI. But yeah, I think you're totally right to emphasise that the other side of that coin is what can you scale that was just previously impossible to scale? And this is an interesting one. Can you describe a little bit more what the humans do? So let's say I'm breached. Did he set the scene in terms of, like, who your ideal customer profile is and how it comes to their attention that they've got a problem and then who they go to and how long it takes these people to engage. And then if I just were to sit there and watch what they do, what are they doing? Are they just like grepping through logs, kind of using the intuition they've built up over time? Or what is that sort of story? And then again, we'll We'll switch to the future state story next.

Yeah. So to make this concrete, so we're currently focused on what's called business email compromise. More and more intuitively, it's just email-based cyberattacks. And broadly what this is, or like the common modality here is an attacker has somehow got access to an email account. Maybe they've sent out a phishing email that someone's clicked on. Maybe they've got access to your credentials some other way, and they're trying to use that access To trick people into doing stuff like sending them information or sending them funds. So this can often is financially motivated, but can also be quite different. So like when the DNC was hacked by Russia in 2016 was an email-based attack. And so you know we asymmetric. So we come in and we we respond to these attacks, as do other folks cybersecurity folks. And what this generally looks like. Is okay, you come in and you pull down all the email logs from an environment. So the the person, the enterprise has been hacked. They might be using Microsoft. They might be using Google. And you look through the logs, and first of all, you're trying to figure out like how did they get access? Like when did they first get access? Did they then escalate their access in various ways? Did they gain access to another account? Something like that. And very concretely, it's like looking through email logs and trying to be like, okay, this is like a weird location to be logging in from. And maybe like the user was in 2 places at the same time. That's impossible. Um, so this is— this suggests something, something is suspicious here. So you figure out what, how they got in, then you figure out, okay, what did they do? You're again looking through the, the email logs, be like, okay, did they send emails? Did they read them? Did they delete them? Often people will set up like automatic inbox rules, so like automatically forwarding emails, automatically deleting invalid emails. Are they looking at files in the Google Drive? So you're painting this very comprehensive picture of what happened and also whether the attacker is still in the network. And then finally you're looking at, okay, how did this happen? Was it a phishing email? And at this point you also sometimes look at the email bodies themselves. And so you have a pretty clear sequence of things you're trying to do, but you're doing this in this sort of very flexible human way of like, okay, this looks kind of suspicious, this looks funny. Let me pivot off this bit of evidence. Let me think what comes next. And so it's been hard up until very recently to actually automate this in a meaningful way.

How much does access to, like, if you're running your own enterprise email server versus you're like a Google customer and your employees have Gmail, how much does that change how this goes down? If you're a Gmail customer, you obviously, or I would presume, you can't just get that level of access, right? So Do they have a team that you work with to try to resolve that sort of stuff? Or how does it work when, when you've got like a SaaS provider like a Gmail?

So most tenants, like a user tenant, will have a bunch of logs that you just use. So actually, they do have all the logs you need. You just get admin privilege to those, to those accounts, and you can go from there. The provider does actually matter a lot. So for example, there are far fewer Google compromises of emails than Microsoft. But I think at least an order of magnitude. So big differences based on how they've configured the environment.

Do we know what causes that? If it's that big of a difference, why has Microsoft not managed to close that gap?

Yeah, I can't recall the specific reason, but it is this very tractable thing they could be doing that they just have not been doing for reasons that are kind of baffling to me. I guess it just doesn't really affect them in this big way. But yeah, I know some folks who have considered working in the space and building various bits of tech to help deal with these attacks and who were like, now probably we shouldn't do this because Microsoft's going to solve it one day. But yeah, up until now, it's not been the case. Yeah, that's funny.

Okay, so how do we get AI to do this? And where are we right now in terms of how good is Claude code and where does it fall short? What are you building to make sure it actually works?

Yeah, so I can tell you a bit about what we're building in this context. We are a full-stack AI digital forensics and incident response company where the mission is to accelerate AI cyber defense, in particular by trying to automate this field of digital forensics as quickly as possible. And we're doing this broadly by having these human AI teams doing these kinds of investigations. And so we're currently focused on these email investigations. And so concretely what this looks like is we go in and do the kind of things I was just talking to you about, but on this AI platform we've built that ingests all the logs. Broadly, we have an agent that will do a first-pass analysis over these logs, and then the human investigators will click through and basically try to do something like a QC most of the time to just check the quality of the reasoning. And in a minority of cases, they do need to meaningfully change what's been done. And so based on what we're seeing in these initial cases, And for context, these email investigations are on the much simpler end of investigations. The models are pretty good already, even without doing anything fancy, without training on specific data, without anything like this. You can get maybe something like 90% accuracy, something of that order of magnitude out of the box. That is super helpful for speeding up the process of this investigation. It's totally insufficient for actually automating this work. This is a context in which you need very high accuracy. for various reasons, which means you do completely need the humans. And it's also an area where the nines of reliability just matter a lot. So for the foreseeable future, and certainly as we get to more complicated kinds of investigations, it does feel to me like this would be an area where there's this long tail of needing humans to be around to really push up the nines.

Okay, that's really an interesting kind of possible production possibility frontier that I'd like to understand better. But just as a little context, I think that this is going to play out, I suspect, in very different ways in different domains. And I'm always kind of like, well, geez, if AI can do 90% of the work, how much does that leave for humans? Is it 10% or is it still like— Can they finish it in 10% of the time that they would have had to spend, or do they still have to spend half the time that they would have had to spend, even though in some sense 90% is done? You can imagine that last 10% taking longer. And then, of course, there's also the question of how many nines do you really need, or can you even measure in a given context? And then there's the question on top of that of, like, how much latent demand is there for different kinds of services depending on how much the overall price can come down based on how much can be automated and how much more productive the humans can be and whether or not they can hit key reliability thresholds. So my usual kind of somewhat tongue-in-cheek way of saying this is like, on the one end, we have dentistry, where I do not want any more dentistry, no matter how cheap it gets. I want the minimum, ideally zero. And it's never something I want. And making it cheaper doesn't really entice me. On the far end, you might put something like massages, where I would potentially get a massage every day if it was close to free. I have limited— I mean, I guess one of the big bets you have is that there is a ton more demand for this than is currently served. How would you think about the thresholds that matter in terms of the ratio of human productivity that would then enable, subject to certain reliability thresholds, that would then enable the vast explosion of this market that you're obviously trying to unlock?

Yeah. So I think a couple of ways to think about it from the perspective of just applied reactively in the context of incident response, that I think is a lot is not of the form that the demand will hugely grow here. Or rather, it's not of the form there is a bunch of latent demand. I do think demand will grow because there'll be a bunch more AI-enabled cyberattacks, but it's not of the form people, if they could have more of this, they would want more. It's just like, after you've been— you're bleeding, you're hacked, you need to deal with it. That's basically the demand. On the other hand, if you get to this paradigm where you're using this sort of investigative reasoning but applying it proactively, then I think it sort of ends up substituting for the current approaches to doing detection in cybersecurity, which are not at all based on this digital forensics type approach. And it's a much, much bigger market on the order of a few hundred billion, something like this, maybe $500 billion. And so I think as a sort of like a bottom line here, at minimum, you're substituting for this existing huge market of detection. Then there's additional question here that I think is sort of an open question of If it becomes really cheap to get high levels of security, like much better than what you currently have in protection, what is the demand for that? That, I think, is more uncertain and depends probably a lot on the type of actor. There'll be some organizations where it's just, you're just doing, you're getting what you need from security and it's like a meet the minimum bar thing. Then there'll be other organizations, say like AI labs who are trying to protect really sensitive IP or national security agencies and governments. who have this much higher demand for very high levels of security. And I think those are the areas we should expect this additional demand above and beyond just substituting for the current approaches to detection.

So how do we get there?

It's a great question. So this actually relates to something you brought up before around the difficulty of measurement in this space. And I mentioned these 2 bottlenecks around actually having access to data or context on what's actually happening, what these incidents, what these cyberattacks look like, and then having the people that can actually assess performance. At the moment, the status quo is that it's very difficult, and we've talked to many of the folks at the frontend companies, it's very difficult to make the models better in these situations because you don't have these 2 things. It's hard to build verifiable rewards for reinforcement learning without having a lot more color on what's actually doing, what's actually going on on the ground in cyber, and without having people who can assess these things. And so I think the key to really accelerating progress in this space is solving both of those 2 things. And I think broadly, the approach we are taking to solve this is actually having this whole cybersecurity team doing these investigations all the time. So we've just hired folks from CrowdStrike, from Palo Alto Networks, from lots of the biggest cybersecurity companies whose day job is really just to do these investigations with the AI tooling. And as they're doing this, they're implicitly evaluating the model performance based on the use of the tooling. And then secondly, I think the tricky thing is you need to really just be seeing a lot of the different types of cyberattacks that are happening and off the back of that, building evaluations that are really high fidelity, really realistic. building environments that are really realistic and that you can use to train on. And I think this just doesn't happen by default. And by default, the model providers, I think, will lag on these dimensions. None of the big cyber companies actually are either pushing on this either. It's not like a huge— they don't have a huge immediate incentive to do this. So unfortunately, I think the default is just this stuff lags behind in a way that doesn't apply as much to offensive security, by the way. Because from an offensive perspective, you can just try hack something. You can just do that all the time. You don't need any sensitive data. And so there is this asymmetry.

Exactly.

So there is this unfortunate situation where you can, like, a lot of bottlenecks to improving the defensive side, a lot less apply to the offensive side. And I think what the space needs is basically a lot more companies trying to solve this, like, data verifiability evaluation problem. We're doing one attempt, but I would love there to be many more.

So how do you position yourself in the market to— because I can sort of see if you become the go-to company that everybody knows is going to do the best job with this, then as we see in— I mean, this is kind of a general phenomenon in the AI space, right? Like, the more you— more of the share of the business you win, then you have more access to data and you get kind of a positive flywheel going. So I can see how that dynamic could work once the flywheel starts to turn. But it does strike me that it's probably a pretty hard market to enter. So how are you thinking about entering it as a young company that obviously doesn't have the track record, the sort of the old adage of nobody got fired for going with IBM, right? Like, I assume there's probably something similar in cybersecurity where it's like, well, nobody got fired for going with CrowdStrike. How do you overcome the relative unknown factor and win business so that you can start to accumulate this data and get that flywheel turning?

Yeah, the flywheel is, I think, actually especially interesting in cyber. CrowdStrike is an interesting example where about 15 years ago, whenever they got started, they actually also started as an incident response service provider initially before they productized a couple of years later. And so for the first 2 years, they were just doing this kind of services. The reason for that was they could solve this problem of understanding what cyber attackers are doing all the time and collect this sort of data. And secondly, build these relationships with enterprises who they were serving. In cybersecurity, trust is really important for distribution because it's hard to assess how all the tools are working. So for most, I don't know if you've been in a bit of HR software, you can just tell there's a thing like Does the thing pay my employees on time? In cybersecurity, you're sort of taking it on trust. Like, yeah, is this thing stopping attacks? Like, I'm not. Am I not attacked, or do I just not know about it? So, like, trust is a bigger thing. And so they figured out that there was this really strong flywheel both from data and distribution from doing services. And this has remained the case since then. I think what's different now is that up and like up and like quite recently, it's been impossible to to get. AI-enabled services to work really well and get these much better margins than you would otherwise. And I think this is what's creating this new opportunity in the space. So we are initially focusing on these email-based attacks where we have built out this platform that basically takes the time required to respond to these attacks down from— depends on the firm, but on the order of 2 days to a week to do these investigations to a few hours. So like a very meaningful difference here. And this means that we can just do these things much quicker and much more cheaply. And that is proving sufficiently compelling to get a bunch of folks to trust us. And from that initial trust, it'll be much easier to get a lot of different types of cases here. So I think the opportunity is coming, I guess, again available because of this technological shift where in this context incumbents are just not adopting as quickly as startups.

And is that just the purely sociological phenomenon that we see in many places of incentives aren't quite there? Who wants to take a risk? Maybe they're billing by the hour or whatever in the first place.

There's some of all of that. Billing by the hour is part of it. Another piece that's more subtle is, on the face of it, it doesn't look necessarily like this massive opportunity. Like, it's a pretty big market. if I recall correctly, it's like a $40 billion market of this sort of services. But from the perspective of, say, CrowdStrike, who is, what, I don't know, close to a $200 billion company, I think, and the majority of their revenues are coming from their detection products, which are not these sort of digital forensics types of services, it's like relatively small fish. And so it only becomes this very compelling, this comparatively compelling thing to this other big pie If you have this view that, yeah, okay, we're actually going to get to the point where we're meaningfully automating all this field, and then this will change actually not just how these forensic investigations are working, but how detection is working. And that requires you to really take AGI seriously and to really be like, yeah, okay, this work that at the moment is just entirely humans is in pretty short order going to be automated in a meaningful way and will change how all these other bits of cybersecurity works. And most of the players in the space, I just don't have that belief in general. And I've spoken to some of the folks who I know who work at the intersection of AI and security. The security people generally don't— they're by nature skeptical people, and they don't really believe in AI in this big way. And so I think part of the opportunity is just like, do you really believe in AGI and have thought through the implications of that? And I think most of the big players have not.

Yeah, fascinating. Okay. Can you tell a little bit more about the jaggedness of models? And I guess I'm also interested in, of course, we've got the age-old debate of proprietary API models, which I would presume are the best at these tasks off the shelf. But then you've also got, if you're using open weights, then you have the ability, of course, to fine-tune or modify however you might like. What's the mix? What's the model mix look like? And how much do you think that this is going to be like? How much value is in the harness versus how much is in the training data that you can create? And how much of— I guess there's also another question of, like, training data can be monetised or can be commercialised in multiple ways. One is you could sell it back to the model providers. And I think that's, from what I hear, increasingly a very good business. Or you could try to make your own models that outcompete them, so you have that asset. How would you describe what's working today, and what is your strategy to not get steamrolled by just better and better frontier models kind of winning everything?

Yeah, on the jaggedness, I think there is actually this sort of underrated opportunity here to to defensively and intentionally accelerate capabilities in various ways. So one intuition, Pompere, in the context of alignment, for a long time, AI alignment, people have talked about the automated AI researcher being one important part of the way we're going to solve alignment. There hasn't really been similar analogous things in other fields. And I think part of the reason was just with pre-training and this very strong generalization, it was hard to see how you could accelerate in this intentional way rather than just across the board. That, I think, with the jaggedness we're seeing, is no longer the case. And the implication here, I think, is that actually you can go out and pick various areas that you think it might be important to harden the world, like defensive cybersecurity or biodefence or bits of ASH, for example. And you could very intentionally curate the datasets, the environments, the evals that you need to sort of pull out the jagged frontier in this specific direction. And people don't seem to talk about this very much. I think this is a mistake. It seems like this really huge opportunity, I mean, for many of the same reasons that AI is going to be so transformative in the first place. Like, if you're a subject matter expert in one of these areas, and instead of doing bits of work yourself, you can just put this expertise into AI systems and then scale it immensely more. That's like an incredible thing you could be doing. And yeah, it strikes me that very few people are doing this. And a few reasons. I think in some contexts there's like, in the context of cybersecurity, for example, people often have this idea that actually it's just inherently dual use. That's, I think, partly true, but also overstated. I saw Sam Altman tweet this the other day. There are some areas like pen testing and vulnerability discovery That are in fact, yeah, they're just dual use. You can patch holes or you can exploit them. But this is not, you know, cybers is a vast field and this is not true of all areas in cybersecurity. And so digital forensics, for example, I think is very much not like this, where fundamentally what you're doing is trying to detect whether stuff is broken into a system. You're asking, is there evidence that an attacker is here rather than, can I break into this? And so that's just like, it's a defensive application. And I think there are other areas of cybersecurity too. that of this feature. And so generally, I think it'd be good if there was much more of this kind of stuff out there. On your other question around the models we're using, how much is the scaffold, all that kind of stuff, generally, the way I sort of think about this, at least from how we as a company should be going about this, first, we should just see what we can get for the low-hanging fruit out of the box. Pick the low-hanging fruit. We've tested different models, see how they go. Build minimal scaffolds, see where the performance is, make sure we have sophisticated evals that are working well here and that we can really tell what's working, and then only then consider doing some more fancy stuff like training our own models or things like this. At the moment, we are at the eval stage of that. And on these simple incidents that we're currently working on, just models out of the box with some scaffold is actually getting pretty far. Again, we're not so far down the lines of reliability, but it's actually unclear how much more effort it's worth from us to get them good at these specific email-based attacks. Going forward on these other kind of incidents that are way more open-ended, long horizon, I'd be pretty shocked if the models were as good or anywhere near as good. And I think that's an area where all this differentiated access to data that we use as blueprints for evals and so on. and the reasoning traces of our analysts using the platform that I think would become this very differentiated and valuable asset.

And do you have an intuition at this point as to whether you license that data back to model developers or keep it for yourself to try to make your own specialist model?

Yeah, both are possible. And you could also imagine doing this sort of iteratively where you have some stuff that you've done in the past that you're then sharing I think the benefits of sharing, at least from a mission perspective, are pretty significant in that if you're getting the foundation models better at these tasks, you then get this immediate insane distribution where you're deployed across huge parts of the world. You're naturally deployed across AI labs, across governments, some of the most important areas from a security perspective. And also, yeah, they will be big contracts and relative to other data providers, just the fact we're seeing all the cybersecurity incidents all the time does make it quite differentiated. On the other hand, is it giving away some sort of proprietary thing? Possibly. But yeah, I think it's sort of an open question. But there are certainly big benefits to sharing, not the data per se, I would say, because that's delicate, but sharing the sort of the evaluations drawing on that.

Going back to that question of generalization, I mean, I do think the Obviously, the jagged frontier is a major factor affecting all of us that are trying to use AI to do stuff on a daily basis. And the question of how well RL generalises seems like a pretty big question for what the next couple of years are going to look like. What would you say is the case for your specialists? Are they I mean, I definitely take the point that it's very different to grep through logs versus try to break in in the first place. But I then would also kind of guess that the people that you have doing the incident response would probably be very good at breaking in, right? They would know all the tricks of how to make a phishing email compelling or whatever, right? It would seem that if you had trained as a human, at least on the response, you would be maybe not the most elite on the attack side, but I would expect significant generalisation. So how do you see that playing out at the human level? And is there not reason to expect that a similar kind of generalisation could happen for models?

Yes. So at this human level, actually, surprisingly, this is not the case. where the digital forensics field is actually a pretty distinct thing within security. And the people who do it are a very different community to the offensive white hat hackers, things like this. And so at least at the human level, there's not a ton of generalization, interestingly. Yeah. And I suspect there is just actually relatively limited overlap in terms of the skill sets here. And I think this would also apply to models. Again, of course, at some point we're going to solve continual learning and the models will just get good across the board very quickly. But until we're in that regime, I think there is this opportunity.

Okay, cool. That's quite interesting. Do you have other things in mind that you think are like that? We can get out of our domain pretty quick here, but is there a biosecurity analogy? Is there something that we could be differentially accelerating in biosecurity hardening that wouldn't lead to the same kind of advances in the offensive skill set? Or is this perhaps something that's relatively idiosyncratic just to the way the cyber world happens to be structured?

Yeah, I will caveat that I don't know bio very well, but you could imagine, obviously you'd need robotics for this, but just producing a bunch of masks, and having those ready to go, like stockpiling masks, stockpiling vaccines. Certainly they seem like there are at least some, like the mask example. I don't know to the extent to which the biological research is dual use. And then on, let's say, what are other areas? I would love, by the way, for someone to just go through and catalogue a bunch of important domains and try to figure this out. I could imagine in the context of AI safety, maybe there are specific things here. I'm not sure, but it seems like an important thing for folks to go figure out.

Sorry, I recently did an episode with a couple pioneers in the formal methods space, and obviously there's multiple— there's a general agreement that cybersecurity could become a big problem in the AI future. And then there's several flavours of ways that we might respond to this. The formal methods angle, I mean, it doesn't really necessarily address email-based hacks, but you could imagine a lot of things could get a lot better if the rate of vulnerabilities is just dramatically reduced based on— and the vision that I understand that they have is using formal methods as a reward signal to train coding models such that it's not just that you're taking output from today's models and trying to verify them, but you're also really closing that loop and getting to the point where models should be writing superhumanly secure code by default in a supermajority of cases. That seems very plausible that we will get there, because it seems like the reliability of the formal methods, I mean, that's obviously its core strength, right? So that flywheel should be pretty easy to get turning. How do you think about where you want to be in the cybersecurity? If it's a defence-in-depth type of thing, right, where you can harden your outermost defences and then you can have better forensics when things do get through. In the extreme limit, if the outer defences are totally impenetrable, then the the forensics becomes less valuable, or again, in the extreme, unnecessary. How do you think about that? Is that just never going to happen in your mind, or what?

Yeah, I think more hiding would be great. People should totally do that. But I'd be very surprised if it would be sufficient. A few reasons. One is just for very high levels of defence against, say, nation states, It's just extremely difficult to defend. And it's not just like having no code vulnerabilities is sufficient. There are other ways you can get in. And the strong assumption should be stuff is going to get in. Additionally, regarding these other forms of ways of getting in, generally to be very, very secure, you need to do these things that trade off with your productivity as an organization. So say, I don't know, say you're an you're an AI lab and you have this way of interacting with the model weights that restricts the output, that's kind of annoying, but it does protect against weight exfiltration and things like this. And so in general, you do have this trade-off with productivity that is hard to get around in many areas of security. And so that's a thing that's generally going to stay there. A nice feature of automated forensics or detection is that it doesn't really have this feature. You can just sort of run it in the background and it does increase your defensiveness. And so generally, I think of these things as substitutes. It would be great if you could get to this full, in theory, full hardening. I don't think that's at all realistic. And so in general, they're just like, the more of each you can get, the better.

So wait, substitutes or complements?

I guess maybe both. Well, they substitute in the sense that if you have more better detection, You need less hardening for a given level of security, but also having both together makes you more secure.

Is there—

I've not actually done an episode with these guys yet, but there's a company called Jericho Security. I just talked to them offline a while back, but I have their website up. Next generational security training trusted by the US government. Smarter cybersecurity training starts here. Jericho Security is designed to protect you against today's most advanced email, SMS and deepfake threats. And basically what they do is automate the phishing attacks, more or less, the spear phishing attacks to see like who in your organization is clicking on these links and make sure everybody's on their toes. And I don't know that there's, to my knowledge, when I spoke to them, there wasn't like a, you know, a deep forensics. They, you, when you click on the link, they, they take you to a, sort of we got you page that's like, hey, wise up, dude. You just clicked on a bad link, and here's how you should have known and how you can know next time. But I could imagine that in terms of environment creation or something like that, there could be some interesting collaborations between a company that is systematically testing for finding the soft spots in the human defences, or lack thereof, and what you guys are doing under the hood.

Are there other—

how do you go to market? I mean, do you have partnerships or people that bring you in? Are there other alliances that you have that allow you to get into the room? Because obviously, people need to— this is like crisis stuff, right? So they need to Either find you real quick when they're searching, or they kind of need to have known you, or they need somebody to give a trusted referral. What does that look like?

Yeah, the go-to-market is actually kind of interesting. It's via, for the most part, insurance companies, actually, cybersecurity insurers. So they will insure a bunch of folks who get hacked. And when they get hacked, they have a pre-approved panel of incident response vendors who they will call in to deal with the hacks. And so actually, a lot of the motion is going and meeting the insurance carriers and talking to them and building trust there. And then off the back of that, you end up building the specific relationship with the enterprise. But yeah, a lot of the distribution is through the insurance carriers. It's quite unusual.

The AI underwriting. You may probably know the guys from the AI underwriting company then, I suppose. Yeah. Okay, interesting. Insurance strikes again. Okay, well, let's zoom out a little bit here. in closing and talk about like, what's the big picture look like? You've got this kind of, you know, initial service that's kind of your wedge into enterprises. How do you see that line of service expanding? You know, kind of paint a picture of like, as this matures and you get really good at it, you know, what can we enjoy in terms of additional security? Does it impact my, you know, day-to-day life at all, or is it just something that kind of runs in the background and is sort of a guardian cybersecurity angel watching over me. And what's the time? Can I invest before you sell to CrowdStrike when they finally wake up and realise AGI is a thing?

Yeah, the way I see this broadly, imagine we have just AI systems capable of fully automated digital forensics and much sooner than we otherwise would, because we've done all this work of bootstrapping with humans and building these evaluations and so on. And the implication, I think, is that this completely changes how defensive cybersecurity operations work. Instead of missing this huge number of breaches because there's no one investigating deeply, you actually just have this proactively on in systems. And so you're catching a lot more than you would otherwise be. And the implication here is just that there's a huge reduction in the number of breaches that go undetected. And then hopefully this is just deployed across all the most important parts of the world. So It's deployed across AI labs, it's deployed across the AGI supply chain, it's deployed in Western governments safeguarding things like autonomous weapon systems. And when you look back at the overall effect of AI as it diffused and the effect it had on cybersecurity, you say, actually, no, look, it seemed like this was overwhelmingly beneficial for defenders. That is, I think, possible. It requires a lot of hard work, but it's pretty clear what you would do to get there.

Anything else we haven't touched on that you want to make sure people are thinking about as they try to absorb your extremely AGI-filled worldview?

I think the big thing for me is just, I think we are one important attempt of doing this, like pulling out the jagged frontier in a way that takes seriously what the world will look like when we get these really powerful AI capabilities. But I see very little of that, and I don't see any reason why that's the case. And generally, I don't think it requires any huge leaps of imagination or things like that. It seems just very tractable, very important to do. And I would love for there to be many more such projects across cyber defense, across biosecurity, across the whole space.

Yeah, time is of the essence, it seems. We're due for the automated AI researcher intern edition in just—

Coming soon. Yeah.

Yeah, wild times.

Cool.

We'll definitely continue to follow this with interest, and maybe with your motivation, I'll even get around to changing some of those passwords.

Sounds great.

Alexei Karlyaushin, founder at Asymmetric Security, thank you for being part of the Cognitive Revolution.

Thanks, Nathan.

If you're finding value in the show, we'd appreciate it if you take a moment to share with friends, Post online, write a review on Apple Podcasts or Spotify, or just leave us a comment on YouTube. Of course, we always welcome your feedback, guest and topic suggestions, and sponsorship inquiries, either via our website cognitiverevolution.ai or by DMing me on your favorite social network. The Cognitive Revolution is part of the Turpentine Network, a network of podcasts which is now part of a16z, where experts talk technology, business, economics, geopolitics, culture, and more. We're produced by AI Podcasting. If you're looking for podcast production help for everything from the moment you stop recording to the moment your audience starts listening, check them out and see my endorsement at aipodcast.ing. And thank you to everyone who listens for being part of the cognitive revolution.