AI:AM — Trust and Recovery in AI (June 1, 2026)

Well, welcome to AI/AN. It has been— we are your hosts. I'm Prakash Narayanan.

Hi, I'm Nathan Labenz. Let's do this. After a little live debugging, we are really live.

And for those of you just catching up, we had some serious audio issues just now, and I told Codex to fix it. And Codex fixed it live. So there's that for you.

This is the ultimate in eff it, we're doing it live.

Yeah.

So, well, we've got 17 minutes before our first guest. I wanted to take a beat here just to start with a little reflection on like, why are we doing this? There's obviously so much noise in the AI space right now that one wonders, you know, and maybe it's just my lack of confidence in my own charisma. to carry a live show. But one wonders, does the world really need more live yapping about AI? And I think I want to get your take on this as well. I mean, you really have been the prime mover in terms of pushing me to try to do something that's higher volume and closer to real time. And I've definitely been convinced that as all the cycles get tighter and from the AI news cycle to the model release cycle to potentially entering into a recursive self-improvement loop where everything becomes so fast we can't track it anymore, that at least trying to track events up until the singularity is going to be a worthy pursuit. And I thought Dean Ball, as so often, captured this really well over the weekend when he said, bad news, friends, it's not a marathon or a sprint. It's a marathon that you have to sprint the full way through. So that really felt to me like, yeah, that's kind of what we're doing. We're gonna experiment with trying to sprint through as much, maybe, you know, the full rest of this AI marathon until we get somewhere, wherever we might be going, and see if we can't make that valuable both to ourselves and hopefully to others along the way. I really think That it's gotta be kind of valuable to ourselves first. As I've kind of reflected on like, what would make this a success? I think demonstrating to the world like what a recursively self-improving sensemaking operation looks like that tries to keep up with the actual pace of events is maybe my best 1-minute, you know, or 1-sentence summary, more so than even, you know, making good content or, you know, making something that's clippable. And obviously that's gonna have to be part of it too, if there's gonna be any traction with this, but I'll feel good even if nobody else pays attention, if we're able to lock into a groove where we are getting better at keeping up, making sense of things as we go, and hopefully sharing that too. I think there's a really interesting opportunity to share a lot of the infrastructure and methods that we're developing as we refine them day by day. So that's my kind of philosophical motivation coming in. What do you think? And, and what would you add to that?

Well, I think my expectation, I think for the last 2 or 3 years has been that things will get faster and faster. And I'm a very conventional, traditional investment banker kind of profile. And on the finance side, people completely disbelieve most of the technology. I mean, you see it today. I mean, SpaceX is about to be included in the indexes as soon as it goes to IPO and like this. Huge amount of like, oh, he's a scam artist. It's a scam. Like it's exit, his exit strategy. And I think even funnier when Elon bought Twitter, people were like, number one, he's scamming because he's selling Tesla. They're like, he's such a scam artist. He was selling Tesla and he's selling Tesla at the top. And then they were like, he should take the money and put it in index funds. That was the other thing. And I think there's this huge misperception, I think, and it's really driven, I think, partly by finance, partly by the post-2008 collapse. There's intense pessimism about the prospects of technology, about the prospects of America, about growth. like just intense pessimism, right? And I, I felt very strongly as the, um, as AI started to take off 3 or 4 years ago, I felt very intensely that we're gonna start to see a shift. Um, and as we see that shift, it's very, um, I think it's, it's very like, uh, not unsettling, not unpleasant, but unsettling to a lot of people because they have a very well-defined worldview of what the world should look like, that you should invest in index funds and that's the only thing that anyone can invest in, that no one has an edge, no one can develop anything new, nothing ever happens. There's that meme going around. And I felt like post-COVID, as COVID started to tail off, I felt like things were gonna change because again, in 1908, post the Great Depression, the Great Flu, you had a lot of changes start to happen because people start to move out of their, you know, like they get dislocated from their status quo and they start to look for new things. And I felt like the technology had come enough that people are willing to adopt it. And you need this kind of both the technology has to be ready and people have to be ready to adopt it. And I think that pull-through came during COVID with the start of Zoom calls. Zoom calls were, you know, we've had Skype calls for 20 years. People didn't work from home. But you saw it in COVID, you saw the population meet the technology at the right level that it started to be pulled through. And I felt like COVID speeded us up by about 10 years or so. And I feel like that is happening with AI right now. Like that process of pulling new technology through is continuing. And I think we're seeing that in drones in Ukraine. Drones have also been a technology that's been around for like 15 or so years. The Ukrainian war kind of pulled that through. I think we have a lot of latent technological power that we don't use because it's not necessary. And the status quo is always a safety status quo. It's like, this is not safe. Or there are political reasons, like all those fighter jet pilots are about to be unemployed by drones. It's a travesty that People are still using human pilots right now, right? But they've all been trained. So we're having one last hurrah in Iran where they get to fly their jets. The Trump administration tried to get a trillion-dollar-plus budget for defense passed last year, additional budget. They couldn't because they wanted to allocate all of it to drones. And they wanted to close down bases. And in the US, closing down a base, you need a base closing commission where all the southern senators of sit down and say like, no, you can't remove this base because all of Huntsville, Alabama depends on that one base. There are no educated people in Huntsville, Alabama besides those graduates in the base. So you can't close it down. So there's all of these intersecting things about new technology appearing and dislocating people. And now we're about to see this acceleration where it's going to happen again and again, over and over. very, very quickly. And I felt that this was the year. And I felt at the beginning of the year that this was going to be the year. And as the year went on, it got more and more certain. You know, I have all of these things like that I expect to see. And as the hurdles fall, you're like, well, it's happening. Last year's hurdle was, I expect coders, junior coders to be completely, you know, the capabilities of junior coders to be available in software. This year, the capabilities of, you know, kind of middle, middle range, middle, middle experienced coders to be completely available in software. And as you can see, we just fixed a bug live in like 3 minutes, which would normally have taken a couple of weeks, I think, of identifying the bug, documenting it for the PM, documenting it, handing it over to the tech team, some back and forth between the 2 prototypes, user acceptance. We did that in like 3 minutes, which is insane, like 3 minutes in live. And so I felt that this was going to start happening more and more. And I think we are seeing that. We're seeing that from all over. And I also feel it's not just an AI thing. It's a technology thing. It's largely technology has gotten to such an extent that, you know, we— what, for example, you know, yesterday pancreatic cancer, There's a drug, Revlimid. There's a drug that improved pancreatic cancer survival from typically 6 months to 13 months. Doesn't sound like a lot, but it doubled the survival time. And that drug started off 4 or 5, 6 years ago, and it's built on generations of technology. But it's getting approved today, like this week. oncologists will approach their patients and, you know, they'll start using it. The, you know, people say AI has not done anything for like drug discovery. The drug discoveries that are happening right now, we will see in 4 or 5 years. Um, and, and which, and, you know, given my timelines, I feel like that means human longevity is going to be solved like, you know, like this year, and then we'll see the effects of it in 4 or 5 years. I feel like, you know, even the GLP-1s might already be the start of the longevity takeoff. You You can see them, you know, some kind of 50% of Americans are obese. Obesity, you know, creates a life expectancy gap of like, you know, 10 years for the average person. Um, GOP wants to reduce obesity 50% in the United States. Uh, that adds 5 years, right? So 5 years to, uh, half the population— 10 years for half the population is 5 years for the entire population. Um, that's longevity takeoff if you have that happening every 5 years. That's takeoff. As long as you live, your life expectancy will get, you know, increase every single year more than, you know, more than the year that you live. So I feel like all of these things are happening at the same time. It's very hard to keep all the threads together. I'm not qualified. I think almost anyone— no one is qualified, really. And you kind of have to, you know, do as best as you can. I guess. And I thought that, you know, this is a, this is a good time for us to kind of try it out.

Yeah.

One thing I think is going to be really interesting for us to watch is, and kind of hold ourselves accountable to, is the right balance between keeping up with all these different directions and trying to maintain some point of view or standard or sense of where we are really going or should be going. I think this is honestly a huge challenge for the field as a whole, and maybe talk about this a little bit more after we have our 3 conversations with guests today too. But I had set out this goal for myself of no major blind spots on the AI landscape some time ago, and I think that served me reasonably well for a while. In that it kind of pressed me to identify my blind spots and do various explorations to try to remedy them. But as things are going so broad and so deep at the same time with things like the OpenAI math open problem, major open problem solved, the first one that I have the sense that if you solve that problem as a human, you would probably be kind of a shoo-in for a tenured professorship in math. Now we're at this really strange challenge where it's like, okay, I can sort of see that news item. I can't really go deep enough to understand the true significance of that in a way that a proper mathematician would. I sort of want to, but the opportunity cost of doing that is going to be high enough that I'll kind of miss a lot of other things for sure. And I try to do all those things and just run myself totally ragged, again, we come back to sprinting through the marathon, then how do I maintain perspective and try to shape, to the minor degree that we can, the discourse and the trajectory of technology in any sort of positive way? And I know some would say that that's just a fool's errand in the first place, that these things just kind of unfold. however they unfold and nobody can really control it. I think it's probably hard for us to— certainly you and me, but even humanity as a whole— real control is going to probably prove to be a difficult thing to create and maintain. But I am also still at the same time struck by just how everybody's running so fast and doesn't really have, in many cases, a big picture idea in mind. I don't know what your take is on Ezra Klein and his contribution to the AI discourse, but I thought his column, I think that just officially came out today, was really apt, where he basically said, what are we trying to do here? We have this general sense that this is happening and it's powerful and whatever, but nobody's really put forward a vision of what exactly we want to accomplish. Do we have any sort of public agenda for how AI is supposed to make life better across the board. We sort of have people pointing to promises, but we don't really have a structured agenda or anything that we can hold ourselves accountable to. So much of the stuff is kind of like this, look at this one breakthrough over here, look at that breakthrough over there. If you kind of extrapolate, that'll be really great.

Yeah.

And I'm not so sure if that's going to be enough. I think I'm going to be very interested in my ability and our ability as a team, as we, as we experiment with sprinting through the marathon, to balance keeping up with the day's events, and also hopefully, like, holding ourselves accountable to zoom out and demand better than just, you know, a breathless, you know, attempt to keep up with the feed.

I think, you know, surprisingly, You know, the world leader that I think has best approached this so far with sincerity and intelligence is the Pope. I think I think he has struck upon a very humanistic kind of interpretation, and he's starting to you know explore or defend the idea of what. it is to be human. And I think that's, he's probably like 3 or 4 years ahead of everyone else because I think he sees the existential questions around the technology, which I don't think the political leaders are very well set out to do. I think philosophers tend to be better at this than political leaders. Uh, who have kind of, you know, who have to get reelected. And I think, I, I think it's great that, you know, he, he wrote something. Um, and I think it's great that he started, you know, uh, started the conversation. I don't think any of the other, um, religious leaders so far have, have even tried. To some extent, uh, some religions will not be able to, um, will not be able to kind of analyze it in that way because, uh, like Buddhism, for example, will just acknowledge that AI is part of the universe and so what, right? It doesn't really delve into the question of like, oh, is it a human soul or not? It doesn't really matter in Buddhism. So I think some religions will not even have the— will not even— it's not an interesting question per se. So they don't assign any specific higher value to humanity versus other living creatures, or other parts of the non-living universe even. So, um, so I think, I think, you know, that in that sense, I'm, I'm impressed. So, and, and, you know, thrilled that he, he took a, he took a, you know, he decided to try.

Um, I'll be interested to see if I can be reincarnated perhaps as an AI one day.

Indeed, indeed, uh, indeed, indeed. Um, on, and on that, on, on the note of reincarnation, which is a good note to start, uh, for, uh, uh, for a segment on recovery and resilience, I'd like to introduce Andy Fernandez. He is the general manager for AI and cyber at HYCU, a company that helps businesses protect and recover the data they keep inside cloud apps like Jira, Confluence, GitHub, Microsoft 365, and other systems. His work sits at a practical edge of AI, not how to build bigger models, But how to— but how to— but what happens when AI tools and agents start touching real company data? If an AI system changes a workflow, exposes sensitive information, or makes a mistake across business apps, companies need a way to see what's changed, understand the damage, and roll back safely. Andy's core idea is that backup data may become more than insurance after a failure, and it may become a historical record that helps companies govern AI, investigate incidents, and, uh, recover from, uh, mistakes. So without, uh, further ado, let me bring up Andy on stage.

Hi, Prakash. Hi, Nathan. How are you?

Hello. Hello.

Great. Good morning and welcome.

Can you guys hear me all right?

Yeah.

Perfect. Not sure how I'm going to be able to be either the Pope or reincarnation on this topic, but we'll try.

Yeah, well, maybe give us the, the 101 high-level overview to start. And I'll just prompt you a little bit with kind of my—

I'm having some audio issues.

Not hearing me?

Not hearing us?

I can hear you, Prakash.

I can't hear Nathan.

Okay, we will.

All right, let me, let me, let me, let me take it off the top. And then we'll, we'll We'll fix that. Andy, most people think of backups as something boring, a copy that you hope you never need, but your argument is that it's more than that. Once AI systems start acting inside companies, obviously the agents run amok, can run amok, editing tickets, moving documents, changing code, touching customer records. And the question is not really, can we just restore the data? It's if— can we reconstruct what happened, what changed, what was exposed, what should we roll back? When AI agents start making real changes inside business software, do backups become the black box flight recorder for the enterprise?

You know, Prakash, that's a very interesting problem statement. And we've talked about this in many ways, but I want to allude a little bit to how dicey I would call the situation is today, even before talking about the backup data, because, you know, Nathan, before frontier models came along, there was already a pretty significant challenge in every organization. What do I mean by this? Once SaaS became the model that every single software company wanted to follow, what you saw was a continuous plucking of systems of record that used to live in a data center. When you were an IT manager 20 years ago, you were the king or queen of your data center, and you owned, in a uniform way, how do I protect and how do I recover all of this data in a monolithic state, which is in a virtual machine or in a physical machine. The moment that everyone started moving those self-hosted systems to your Git repo, your knowledge base, to the cloud, all of a sudden, you go from, here's my castle, I can see, I can touch, I can patch, to, well, I hope that all of these vendors are looking out for me, and I'm assuming it's their problem. What we saw was a very different world where all of the systems of record were everywhere all at once, just to quote a great movie. The whole point here is that Even before AI, there was already a heterogeneous environment where you don't know where your systems of record are. Some customers had a great posture in how they were able to protect it, some didn't. All of a sudden, you allow agents to move at machine speed across these applications where no one can really answer the question of, hey, if I clone this repo, if I delete this repo, is GitHub going to help me or am I on my own here? And you'd be surprised how many people can't answer that question, right? So you're sitting at a point where data is more diverse and unclear than ever before. All of a sudden you're allowing, let's call them scripts with personalities, to touch these applications as well, right? And so, you know, the problem statement for Kaushik and where we've been going to is you need a historical index. You need a full understanding of what's happened in your data across all points in time. Because the mistake now is not Johnny deleting something and emailing IT and asking for it back. It is an agent very, very quickly being able to blow up an entire project or expose data in a way that people can't even catch up, right? So this problem statement is a very interesting one today. And we feel that, you know, ironically, the most reactive component of all of cybersecurity backup is primed to be that black box that you called.

So when— what do you think has been with your clients? What have you seen where your product would have helped them if they had had it, but they didn't have it, and then they saw this disaster happen?

Yeah, I'll give you— I'll give you an example, uh, even of how this came to be or where we exist today, you know. When you back up something on-premises, you're backing up a virtual machine, right? It's very, very fundamental. You know, you take a snapshot, you store it somewhere. What it takes to back up a SaaS application is a different beast. You have different levels of APIs, configurations, files, identities, logs to manage. So in order to even be able to protect that, Prakash, you end up having to really understand how that application works at a very, very intimate level. When you back up something like a SaaS, let's say GitHub, you have your logs, you have metadata, you have configs, you know everything that's happened. We've had so many customers where they have deleted something, even before agents, it was just via automations, they were deleting things, and all of a sudden, they had no idea what was deleted. They had no idea where to even start. It's not like these are emails where I can search a person. These are repositories, configs, CMDB. This is not something that's very easy to understand. And also, I have no way to recover from it because you have security tools all the time that do a great job of saying, hey, Prakash, Nathan, something's going on in your Asana. Somebody has accessed your Asana, but that's pretty much it. But you can't recover Asana because you're not backing it up. So, Prakash, it's both customers not having an understanding of what happened nor the means to even bring that data back is where we see the biggest deltas.

So, um, have you seen that kind of disaster happen where the agents kind of went in and just, uh, wrecked someone's repo and, uh, they had no way of recovering it? Have you had—

Absolutely. Yeah, absolutely. We've all seen the, the famous, uh, case studies published online where Being cloud, somebody deleted production databases. But I'll call back to even pre-AI, Prakash, where when people first started using Terraform or CloudFormation, you saw all the time people nuking entire cloud projects because they were so poorly written. The template— there was a mistake in the template. We're seeing the same things here. We see it especially in in-app types of experiences, where if I have a Copilot in GitHub or Agent Force or Rovo and I give it right permissions, All it takes is the wrong set of instructions, and you can immediately go and delete these things. It all depends on how these companies deal with backups. I'll give you an example. Atlassian, they do a good job of backing up their own systems. They'll tell you, hey, we back it up, so if there's an outage, we're good. But, they tell you, if you delete something, there's nothing that we can do to help you. That is your responsibility to back up and recover. What you see all the time For example, in service management world and CMDB world, you see customers write automations, automation goes wrong, and they wipe out all of their assets. And all of a sudden they have no way and no means to recover those assets and the history of those assets, which in many cases you're talking public sector, you're talking financial services areas where you really need control of those assets. And so what we're seeing is agents just acting as a, as an order of magnitude more of risk because of the speed and access as well, because a lot of the times automations were very constrained. All of a sudden, in a software lifecycle development, the agent or the copilot has the same permissions as the developer who's created it, and that's where the risk exists.

How does this differ? I think, you know, can the same functionality be achieved by telemetry? Because a lot of like agent startups, like there's a lot of like startups doing observability for agents right now, and part of that observability is to track Every single step that the agent takes through the, through the, through the, through the systems of the company, through the, you know, VPNs and inside the, inside the network. So the telemetry presumably tells you what the agent is doing while the recovery kind of gets you to return to a point in time or something like that. So do those things work in tandem or can they— does one like subsume the other?

Right now they, they have to work in tandem. It's not a perfect ecosystem. You're seeing the birth of not only better agent telemetry, but better actual agent identity governance, right? How are these permissions being handled? You're seeing all these new tools come in play. What we're seeing, especially in the mid-market, Prakash, is folks are still not adopting these technologies. These are very, very new. So what you have is a former, like, human identity-based Logging and understanding of information, and then the ability to back up and recover that data. A lot of the times, what you'll notice, too, though, is that there is going to be observability and telemetry of a specific application. Agentforce is going to tell you exactly what it's done. However, people are writing these systems across many applications, and what you have to do is be able to understand the logs, the permissions, and the change rate of every single one of these applications in unison. Now somebody is going to come in and become essentially the whiz of that world. But what you're going to experience is a lot of mid-market customers today, Prakash and Nathan, they've got budget for an EDR, they've got an identity budget, they've got networking budget, and they've got backup. There's a whole string of problems associated to data security posture management that many mid-market organizations simply cannot even afford to think about solving. And so our thesis here and our approach is reuse the backup data that you have, which does act as that black box for the organization, and unlock those efficiencies from something that you've already paid for. So, Prakash, what we'll see is someone will become that enterprise observability for agents. It might be an existing incumbent, it might be somebody new, but they will have to work and coexist alongside. And they're going to be responsible for also enriching every other system, whether it's SIEM, SOAR, you name it. They're going to have to also become an enrichment factor. We see backup data as another enrichment factor as well because you get an immutable understanding of what happened. Once you take a backup, you move it offsite, it's object locked. It is impossible to change that. So you have an immutable point in history that you can compare to and also baseline against because our customers are keeping backups for 5, 6, 7 years. We see these as coexisting, but in the mid-market we see a lot of organizations that are going to really depend on that backup data. In order to achieve some of those outcomes.

Let's say, you know, you have— where does your organization or where does your company sit in the chart with CISO, right? Like you have a Chief Information Security Officer. He's got a lot of concerns. He's got a security team. He's got a, you know, cyber— cybersecurity team. He's got an incident team. Like, where do you guys sit in that, in that picture of, of the budgeting of the, of the security function?

So historically, data protection has existed explicitly within IT and CIOs, right? And especially when everything in all systems of record operated as virtual machines, CIOs owned the complete scope, budget, and understanding and selection process of a data protection solution. The moment those applications moved away to as a service and application owners, SecOps, and many other people got in involved, we are seeing CISOs more than ever. In many cases, what they're doing, Prakash, is becoming a champion and saying, hey, we really need to make sure that we include our recovery posture as part of our overall posture. What you're seeing is either a shift or an inclusion of the CISO into these data protection conversations because, at the end of the day, every single organization has had an incident where, despite their best approach to being secure and proactive, Data was lost. The inclusion of the confirmation that you can recover is now within the CISO agenda as well. You're seeing that shift where the CISO is either part of the selection process or even helping with budget compared to a CIO.

Can you hear me now?

Yeah.

Andy, do you hear me? Still no?

I think that's still a no. Nathan, Nathan. We still need, uh, you know, uh, Andy, Andy, can you, can you just refresh your page? Just press the refresh button.

Yeah, absolutely.

All right, let's see, let's see if that works.

Recursive self-improvement.

Yes, it was, uh, we just, we just pushed, uh, we just pushed an update, so I'm hoping, I'm hoping he manages to come back in a bit. He might need to log in and log on again. So we— ah, there he is. He's back. And let's get him back. Assign first spot. And there we go.

All right. Can you hear me now?

Yes, I can.

All right. Amazing. So this is— people have talked about building the plane while you fly it for a long time. We are literally vibe coding our own platform for this show. And today's the You know, first day of the new platform. So you're—

And you're our first guest. So that is awesome.

It's great to get—

it's great to have someone in the recovery business because we are recovering as we speak. So that is awesome.

So a couple— I have a couple of like really big picture zoomed out questions. And I guess first I would say, I've lived this value proposition that you guys are putting forward with the— I don't know if you pronounce it AIR, A-I-R.

Yeah.

I've lived that just myself over the last few months because I've created essentially a local backup of everything I've done for the last 5 years digitally. All my emails, DMs, all the calls I've recorded with audio transcribed, all the podcast episodes. It's all in one database now where Claude can search through this history. And then I've layered on monthly summaries and annual summaries and wikis on top of that. And it's really getting extremely good. It can find really whatever I need it to find. And that is a paradigm shift in the way that I relate to my own history. I find it's changing my behavior even in ways, you know, where I'm like on a call, I might ask a question now that I previously wouldn't have asked because I didn't think I could really understand or, you know, make use of the answer. But now I'm like, I'll log that, I'll get that answer, I'll log that answer, and in the future I'll be able to access that answer. So It is definitely something that I feel you can imagine now extrapolate that to the enterprise scale. I see the value is super clear, but a couple of big questions I have. One is around how people should think about this economically. I mean, right now we have kind of a pinch on GPUs, right? The cost of GPUs has gone up. There was just announcement from Palo Alto Networks that Mythos has done this incredible work for them, finding way more bugs than they were able to find on their own, which is obviously super valuable to them. They also spent $1 million on tokens to make that happen. So I think people are going to be looking for ways to trade off. Is there any way I could invest in something else that's complementary to inference so that I can maybe save on tokens and have some other way to capture that value or what have you. Do you guys have a sort of way that you think about that where you're like, buy more disk space and that way you can maybe save on tokens now? Like, can you afford to run a little leaner or more dangerously, so to speak? You know, maybe have less defensive agents, you know, constantly monitoring all the agents doing the actual work and realize savings there because you move that investment to disk and sort of have the ability for recovery later? How do you guys think about the economic balance that we might end up with?

So, Nathan, even pre the token maxing world that we live in right now, as the kids say, what we noticed was this dilemma already existed in cybersecurity in the sense that I'm the CIO of a healthcare company with 2,000 to 3,000 employees, you would be pretty shocked to see what those budgets look like. And you would be pretty shocked to see, you know, the staffing. These folks are heroes in some of these organizations.

Right.

And they have a budget and they can earmark things for the fundamentals, the networking identity, data protection systems, things that have been written in policy. Right. That you have to have.

Right.

But there are so many adjacent problem statements that have not essentially been solved. So you see all of these other smaller security industries pop up. Identity governance, UEBA to understand insider risk, DSPM. But what all of them are doing is essentially creating their own connection ecosystems. They're saying, hey, we're also going to connect to your systems. We are going to apply our logic. We are also going to take copies of this. We're going to add it to our data warehouse, and then we are going to exert those costs upon you. When you're a CIO of a company like that, your options out there are either What does Microsoft give me for free, essentially, in Purview, or I've got to go to the Bugatti dealership? And so that's the dilemma that a lot of these mid-market firms go, both from a, you know, I don't have the war chest on cyber and I don't have the staff to even deploy and implement something like this as well. So where we come in is we say, look, you already have a connection point to your data. You already have that. You have already invested in this. All we're allowing for you to do is apply agents and information and logic to be able to understand and derive the same intelligence that you would from a real-time security connector for adjacent products as well. So we're already seeing that. We obviously design with the most efficient mindset. You know, you don't have to deploy an LLM on everything. You want to make sure that you leverage small language models, that you leverage all these other toolsets available to you, because the goal is The customer's already paid for this asset. It used to be a very boring asset because it was just what you dusted off to recover. And all of a sudden, this asset now can allow you to subsequently replace a lot of those secondary security tools that you're paying very, very high premiums for, and that you're also having to manage an entire Rube Goldberg machine of systems in order to understand, hey, did somebody delete stuff and when did they do it? And like, what's their passport number inside of that?

Right.

Those very basic questions are incredibly expensive at the mid-market level. And so it's changing kind of the, the mindset people have about backup data from a, you know, kind of, hey, this is a sunk cost, something we have to do, to an operational asset that I can use every single day. And guess what? I already paid for it because I legally have to have them.

So one question for me is that, you know, this sounds like a very classic, like, wedge strategy where the backup is kind of like the wedge and then you're using the backup in order to expand in the enterprise to other functions and absorbing functions that would normally be carried out by other pieces of software. And so, you have a core product, which is the backup wedge, and then you go in, then you say like, okay, let me absorb this, and let me absorb that, and let me absorb, you know, one by one. It's kind of a, you know, land and extend, right? Land and extend kind of strategy. To what extent You know, are you— is this strategy really being powered on your end by increased coding capabilities because of agents? Like, do you find because internally your teams are using, you know, coding agents, they are— they have become more ambitious on attacking other, you know, adjacent parts of the stack?

No, absolutely. It's been Obviously, everyone has experienced this catalyst in software development because of what we have available in the tooling, and we, like every other organization, have been able to do the same. What's a little bit different about this is, you know, you have— let's grab a couple of applications, right? You have a backup of GitHub, of Jira, of Okta, and, you know, even Entra ID. And when you pull that together and you essentially have a natural language layer on top of that, It actually becomes less about, hey, let's go build something new for this new insight, to what can customers glean from this data based on the queries that they have. And so this, Prakash, has really reversed the dynamic for us from, hey, you know, for this next sprint, we're going to go and understand and build this new agent to learn this, where the customer is actually creating their own templates, creating their own insights, which, you know, one query is something that they would have gone from another solution. And then they're informing us in this marketplace of like, hey, this is what we were able to find in backups. Like, hey, I'm worried about exfiltration or exporting of information in Box because people leave to a competitor and take all that data with them. And I'm going to build a workflow for that. Or, hey, I'm going to be able to monitor for PHI in this specific application, cross-reference these people leaving, right? So essentially we've done all the hard work with the data. Now it really becomes how quickly can customers get that information? And so it's flipped the product management dynamic from here's the next 10 insights we think people want to glean to here's what the market has created by themselves that we could either mature or expand into.

One more for me before we run out of time on kind of where this is going from a social contract level. I'm always interested in do we need a new social contract? One of the things that I found when I was preparing my own highly searchable version of this original raw kind of backup data, right? People sent me an email. They never thought I was gonna look at it 10 years later, let alone that I was gonna put it into a system that an AI can look at for me 10 years later, right? So I kind of had to think for myself, like, what was the social contract that I had with all my— all the people in my life when they sent me stuff? And am I in some way violating it or putting them at risk as I make this information more searchable. And I've come up with, I think, some interesting ways to try to square that circle where I have one full access version that just like runs on my laptop and is not autonomous. I call that the high access, low autonomy version where it's only searching for exactly what I tell it to search for. Then there's the more autonomous agents that run over on the Mac Mini, and I had to create, or felt that I needed to create a version that stripped out anything. I tried to get the AIs to use the heuristic of only put for these more autonomous agents the things that it would be okay for me to tell a human assistant if I were kind of trying to onboard a human assistant effectively. Now, you think about that in the context of a big business, right, where people have been working for years. And clearly it's one thing to say, well, we're going to monitor all the agents, but there's a tonne of data there that represents all the stuff that people have done going back years. And I would imagine mining that is going to be extremely valuable and something people are naturally going to want to do. But I wonder what you guys are thinking about and hearing about and hearing from customers in terms of, you know, employees probably weren't thinking that every little action that they made going back years was going to become like really readily searchable. So are they ready for that? Are they going to protest that? Is there going to be some sort of filtering step that is going to be needed to kind of put everybody into a reasonably comfortable place? And in the future, do you think we just end up in a place where we just have a lot less expectation of privacy at work? Or, you know, what is the kind of equilibrium that this settles into over time?

Yeah, for us, you know, there's a couple of very specific levers that are needed that a company has to have in order to be comfortable with this. One is you have to have very, very specific guardrails about who can query what, not only at an application set but a permission set. And what's good about this is that customers have all that very much defined, right? So like, for example, an IT admin today experiences that already with Microsoft 365 and Purview. So if, you know, you guys decide that you wanted to check an email at a certain point in time, your organization grows really heavily, there might be an IT manager, Nathan, that is not allowed to search your emails, and the system already allows for those guardrails. What customers need and what we have from day one is, do I have those guardrails in place? Do I have regional guardrails? Do I have application guardrails? Then do I have user and permission guardrails as well? That's table stakes for any of this because those dynamics already existed. They just didn't exist at the volume and speed that we are experiencing today, Nathan. But decoupled from IQ, that's a very, very important question that we're going to have to answer because every single data point What I'm noticing as well is there are data points that a human would ignore that are extremely valuable to inference systems. You want to be able to keep as much as possible. For us, we're going to allow for the customer and the organization and the compliance standards that they have to have the most safeguarded, guardrailed experience possible based on the permissions and systems that they already have today. But it is a fascinating question that I think is going to change over time.

Just to give you a flavor of what on Twitter at least has been talked about, Patrick McKenzie had this tweet thread on how if you were a bank and you're a banking CEO, you realize during the moment that there are probably some social things going on Which are not going to be looked at in good light 10, 20 years in the future. Whether that is if you were a German banker in the 1930s or et cetera, or if you were a Swiss banker in the 1990s and 2000s. And so what you come down to is You really don't want to keep records too long because you don't want to be called up today and someone says, look, our client is going through a divorce and they want records dating back to 1975 on all of the transfers that came into the account and they have a court order. And so to avoid this thing from happening, banks kind of collectively kind of forget at the 7-year mark, and they kind of destroy all the records so that you can't find them. And so if you have the records, if you have your own paper records at that point in time, you're welcome to bring them, bring them there, and they will say, okay, you have the records, and so there must have been something there. But they are not going to keep records for you, uh, because of these reasons, because they know that, you know, in the future someone might find something that was done in the past was incorrectly done and would seek to correct the record. And they don't want to be responsible for that at all. And that answering the collective kind of like, what is the collective responsibility? The banks know that they have this collective responsibility, that they have this collective power and that they purposefully forget after 7 years. So yeah.

You'll find those retention periods already defined in a way these banks have, or any organization has, a retention period that they follow. There's also another major reason, and the other one is clarity of insights. There are going to be certain application sets and datasets where you want to delete things after a while because it becomes repetitive or the wrong information is provided, right? So, you know, for example, you deploy an app with a Copilot on something like Confluence, and Confluence, you keep all of your documentation, you keep a lot of your instructions, your KBs, things that you're building. There's going to be a point in time for a feature set that you're going to want to get rid of that because you do not want an agent training on it anymore because you've either fixed it, addressed it, moved on. A lot of the times there's the ethical compliance layer and then there's also the, we need to keep our data hygiene accurate and therefore need to adjust our retention policies within this as well. There are a lot of dynamics that are going to drive people to have just the right amount of retention history. and data that they can learn from.

Absolutely. Andy, I'm gonna, I'm gonna cut you off there. Thank you. Thank you for joining us. Uh, you are our first guest on the new AI AM, so, uh, happy to have you.

It's a privilege.

Cheers.

Look forward to having you back on sometime.

It's been a pleasure. Thank you so much. Cheers.

And, um, so we're gonna rush through, so We're going to rush through here. We have our next guests, David Villalon and Manuel Romero. They're the co-founders of Mysa, a company building AI systems for serious enterprise work, the kind where a good-looking answer is not good enough. Their focus is on AI digital workers. This is going to be interesting because Elon has also been talking about AI digital workers for a while that can carry out multi-step business processes in areas like banking, finance, energy, and infrastructure, where companies need clear evidence, oversight, and a record of what's happened. David brings the CEO and company product frame, uh, why enterprises need AI systems that they can actually trust in production. Manuel brings deep technical credibility from the open-source AI world, including work connected to major open model efforts like Bloom and StarCoder. The core MISA argument is simple: enterprise AI cannot just be a clever chatbot. It needs to follow rules, use tools carefully, show its work, and leave behind a trail that people can inspect later. That makes this conversation timely as companies try to move from impressive AI demos to systems they can safely use in real-world operations. Um, it's going to be interesting to see because we have heard a lot about all these firms who are only doing demos at this point. So it's going to be interesting to see whether there have been any real substantive deployments. David, Manuel, welcome.

Hello.

Great meeting you. I'm here in the same room. So, Manu, maybe we will have some echo. Thank you so much for the intro. I'm happy to tell you a bit of the thesis that it's a bit contrarian to the wave. And maybe we also create some interesting discussion here. I would like to also hear your perspective. Because from the intro that you made, on the case on, unless you have some questions early, I'm happy to tell you more on why do we say that for regulated industries, we are doing what comes after agents. Like agents are a thing of the past.

Let me ask you, when you say an AI system should be accountable, In a banking or regulated framework, what does that mean in practice?

I think that this is related to— accountable means that it's able, it's capable of owning an outcome. And that means, owning means that what makes the error is not the human, but the system, and the system holds the responsibility. Accountable means at the end that you know that when something goes wrong, the system will be able to tell you. So you don't have to freak out whenever there is a problem, an hallucination, but you don't have those. You have someone that wants to look at them. That is mostly what accountable means for us at the end. That is owning the responsibility of the delivery.

So when you work with I believe Bank Santander is one of your investors. I'm not sure whether you have a working relationship with them or that you can disclose, but when you work with a banking partner, what is the first task that you take on? Is it like loan underwriting? Like what business process do you start off with?

We have, it depends also on which line of business we are entering that new client. So we are entering commercial banking. we might go with lending and it can go with mortgages, auto lending. If we are going consumer finance, I mean, if we go towards the direction of wholesale banking, we end up in trade finance or also adverse media. If we go to wealth management, it's fund-based performance reporting. Like we do have moved the needle in multiple places already with Santander. It's, yes, also an excellent example where we Have scale in a lot of countries with them. And we leverage some of that also to get to these different business units. And we literally go to them with problems that we know are unsolved with the cases that I just told you, for example.

So when you do a deployment, you know, a lot of times when you deploy these AI tools, you come across, you know, it maybe works for like the first happy case, right? You take one use case all the way through, happy case. And then immediately as you deploy, you run across the edge cases and they start piling up. How do you deal with that process? Is it like, do you try to design ahead of time? Do you go through a long kind of like, do you sit down with your partners and kind of like— Walk through all of the edge cases first, or do you kind of deploy first and as the edge cases pop up, you fix over time iteratively?

It is more that we come with the use case that is created by the business user. Like there are 2 scenarios. Is this the first time that we do it or the client does it, or this has been done before? If it's the first time that it's done, it's created from scratch using natural language and then you need it to face reality, to start having edge cases. Because if you go to any knowledge worker in the world and tell them, tell me how you do your work, they will tell you happy path plus the most recent pains that he had in the last 2 months. But until he does not face a new situation that he faced 3 years ago when he just joined the company, he won't be able to tell you. He won't be able to tell you the insight that you need to know about how to manage the situation. So the best way that you can do is to expose these systems to production. And in that case, the only thing that you can do is to not involve the human with just check everything, because if not, they don't have the time and they don't want to do it. You need the system to be able to flag those situations and say, this is a corner case, this was solved this way, it's correct or not. And then using that as a way to create a feedback loop.

So that sounds like you have a kind of human in the loop for exceptions, and then the exceptions— and then you have exception handler who builds kind of new workflows for every exception case that they come across. Is that— would that be correct?

Yes, in a way, but we don't use workflows. We don't have boxes with arrows, if-then-elses. Those— we don't have that from a UI, from a UX, or from even a mental model perspective. Because the problems we solve cannot be modeled with workflows. And that's why RPA did not solve them and why the new tools are not solving them. Because they are not linearly, like literally every problem, it's almost a new problem space. You just have to play with the hand that you have to, like with the rules that you have to solve it. So it would be more like we create know-how that we give into the worker. And once we give the exception, And sometimes the exception does not deliver any human in the loop. Like, it's just when things are flagged because it did not work or because it made mistakes, leveraging one of our unique advantages, which is that our system is fully reproducible and hallucination resistant.

I wanted to ask about workflows. It's funny that you mentioned that because we're my late night session last night went with Claude Code was creating an automatic upgrade skill for my increasingly sprawling personal AI infrastructure. And I had this interesting experience where I first created the upgrade skill and then I used it and I said, okay, go out and look at all my stuff and tell me what I should upgrade first. And I particularly said, and I wonder if Anthropic's new workflows thing is gonna be applicable for me. And sure enough, it came back with like the most recursive self-improvement thing you can imagine, which was to say the first thing we should upgrade is the upgrade skill, which happens to be a great candidate for the new workflows feature. So I'm in that process right now of kind of moving more toward workflows. And I can't say I'm, you know, with this new instantiation from Anthropic, I don't have a lot to say, you know, it's working, it's not networking, what its pros and cons are. But it's striking to me that you said you don't even have workflows as kind of a mental model. That seems to be at odds, at least with this Anthropic launch. So would you critique their launch? Do you think something's off about that mental model? Should I go revert my upgrade skill migration to the workflows paradigm? Where do you really disagree with that direction?

I'm happy also to hear Manon's side. I'll tell you my most production— like, I tell you the product vision and we can also go deep on the workflows topic. By the way, I don't critique it. It's simply that the day that you start paying the tokens at that cost, you might not want to click that workflow button twice. So I tell you what happens. It's a mental model problem. Because if you think about workflows, you are constraining your thinking into a process. And the problem of that is that the business users that do know how to make the task cannot translate their task into workflows because indeed there is so much variability. There are no happy paths in knowledge job tasks. There are no happy paths. A happy path of I read a document and I put it on a database is not a happy path because the document can be in Spanish, in Chinese, Colombian, it goes with a passport. Now the passport, it's not anymore like— imagine if you had to think on a workflow to validate 2 documents. You would say, no, it's easy, you read the document, you create a JSON, and then— but you, if you really want to make a workflow, which is what people think, you are constraining by yourself the capabilities of what these systems can do. So instead of talking about workflows, which is a process, boxes with arrows with a lot of if-then-elses and some now magical boxes called AI agents that become black black boxes that we're not sure if they will do the same thing twice, but we put them there as routers or as intelligent conditionals, no? And then, um, we think more into the framework of delegation. Suppose that you, you want to automate— or not automate, you, you want to stop managing your calendar. You have 2 ways. You can create a workflow Uh, so good luck managing all that. Or you can hire a human today, no? And you would delegate into that human that. And delegation means that you expect that person to keep learning. You expect that person to be able to do new circumstances because it knows already how to behave from a general perspective. And you wouldn't have to say every day you open the email and you click the button read. it and you do this and then you put the label. You don't— that's a workflow, but this is not how we think. So the problem of workflow thinking is that it constrains what this technology can really do once you solve the problems of cost, reliability, and reproducibility and hallucinations. While you solve that, the problem is that people are still constrained with chatbots and with this if-then-else thinking that constrains the— No, no, no, you really can do this thing end to end. So that's why we challenge, not challenge, we literally ask, if you ask our customers, they will never ever say the word workflow, which I think that it's one of the biggest things that we were able to achieve to scale this.

Yeah, hi everybody. Workflow sounds a bit like low-code, no? Because you have somehow to structure the parts, no? You have to join the dots. You have maybe to be a bit— to have a bit of knowledge, no, of technical stuff, no, how to connect some services and so on. And what we wanted is like, we want people to have the same experience that when they onboard a new person in a new business, no, same experience that you say, hey, this is how we work, this is what you have to do. You give some instruction, you give some know-how about how to do the things, you tell him or her about some edge cases or whatever is happening, but you do it in natural language. You are not thinking in arrows or workflows, or you have to do first thing and then this, and then you have to— So we saw it that way because it simplifies a lot for the regular user, the business user, the things. And under the hood, we translate it to, let's call it, a program for AI. We give a structure, we give a very well-known structure that we feed to our system and it executes step by step generating code traces, no chain of thoughts, code traces. The traces are being retrofitted to the LLM and the LLM decides what the next step, the next instruction and so on. Until the job or the task finishes. So that way we have this kind of auditable, reproducible output that is so valuable in regulated industry, because otherwise LLMs are black boxes. You don't know what's happening inside their weights. You cannot trust chain of thought. There are many papers out there that say, hey, you can't trust chain of thought because we reward models when we train models, we reward models because of the answer, but we don't check the path to the answer. And I'm talking about foundational labs. They reward if the answer is correct, but the path, they don't care. They assume that is correct. But in this kind of regulated industry, you know, this is very important. Well, also for me, my daily job, I don't want this kind of hallucination or this kind of stuff. This is why we do things this way.

So can you Could you maybe give me a little bit more on how it works at the core level? I'm feeling a little tension, I guess, between the idea that on the one hand, workflows are overly constraining. I hear that for sure. But that has been the strategy to create something more deterministic. And then on the other hand, the other kind of strategy is like agents watching agents, and it's just turtles all the way down in terms everybody's monitoring everybody else. But then that has like huge token costs, which you said also, you know, it becomes a problem. You don't want to click the button twice. So how are you— what's the happy place between those? Is it some sort of like domain-specific language type of strategy that you're creating? Or what is the key insight that helps you create higher reliability without Without having to burn zillions of tokens, but also without being overly constrained into workflow-style thinking.

Manu, go on if you want.

Yeah, for me, the key— well, I don't know if we can reveal a lot of our IP, but for me, the key is treating the model as an orchestration engine that the only way to work is to decompose the problem in small an atomic, let's call it instruction, like a processor. I don't know if you are so technical, but you know a processor is executing instruction one at a time step. We force the LLM to work that way and we make it aware that it's well connected to this execution virtual machine environment. And then the only way to resolve the problem is to decompose it in small steps, bake on code everything, every step baked on code, and given the feedback that received from the virtual machine, because it's code, it's deterministic, you can figure out the next step or the next instruction. You cannot, you know, we don't do planning because if you do planning, what if something in the middle fails? No, this, the side effects of our technology is again the traces and something that we call self-healing. Imagine that we have No, we have a tool for reading PDFs. Imagine that the service is not available. With our technology, it's like, okay, I tried this code to access to this service, but it's not available. I can try to install another library. So you have this kind of self-healing. You give feedback yourself in real time. You are building a program in real time. I want to say that we are compiling and executing a program on the fly. Because in classical programming, you know, you program your script, you compile it, and you know exactly what is going to be executed. Here, no. Here it is compiled and executed on the fly, in real time.

And at the end of the day, you have the result of your task.

And of course, you have a script because everything is code, even the docs through commentaries, that is 100% reproducible, no? Because if you do something like python -m script result, you will get the same results again, no? So yeah, this is the way, no, we mitigate hallucination. And we are highly valuable for all these kind of industries where you have to— where you can allow any kind of file, and you have to— you, you, you have— you have at least to have the opportunity to audit, no, your system and to check why, why Why the necessary speed, or what's another one?

So just to kind of ground it concretely, when you work with a banking client, like, what does a deployment look like in 90— what does a successful deployment look like in 90 days? Right? Because banks have a lot of like process, right, that you— in order to deploy, and you start off with You know, most firms, most teams are starting off with pilot projects. Like, what does a successful deployment look like in 90 days? What would have been proven with the bank? What KPIs would they use to judge it? Like, give an example of something that has already happened. You can pick a vertical, you know.

So how is our journey since the day I meet you and the next 90 days in a way? Just for you to know, no bank will close you a deal in 90 days. They might close you a pilot, but a multi-year contract, it will take way more than that. obviously, and it has nothing to do with the product. It's literally the time that it takes for them to make a decision and scale it. But let's put on a scenario of working. The first thing that we do is we do 2 things to move very, very fast. We create an isolated environment on our tenant to start testing as soon as possible with demo data that we have because we have a lot of use cases or start addressing their main problems. So you don't have to pay anything before they have a proof point. That usually takes between 1 day to 1 week. In 1 day, we can prove them that the worker works. In 1 week, that has a performance that is expected. By default, it's 10 times cheaper than agents. But then this is meant to be deployed into your cloud, into their cloud, and even on-premise on machines that you have in a physical place. In order to do this, we need to go through the vendor process and so on. That's obviously CIOs and data protection. And obviously, you know that. We usually— what we optimize for is on our promises, 3 things. A business user will be able to do it or easily involved. We will help you to get it to production, but you will be able to have much better performance than what you have right now trying to do the use case. You'll be able to reproduce every result that it's created without having to call an AI again. So you will have auditing material for auditors that is compliant with DORA, with GDPR, with the European Act, and so on. And last but not least, in the next— in the first 90 days, you won't be dependent on OpenAI or Anthropic once this use case is in production. You will be completely independent of them, be able to self-host it while at the same time scaling as much as you want. The happy scenario is these 90 days we focus on one use case, the one that we chose, that you choose, that has the most impacts. And we go from the beginning, getting it in one day to surpass probably your, or even one week, the results that you want, to in 90 days you have your own model, or let's say more than your own model, your own part of the inference to manage it without being dependent of OpenAI or Anthropic on a scale. And then if you are happy with this, which turns out that our clients are happy with this, we start deployment across countries and prioritizing based on impact expected, which is based on ROI, our ROI, I mean.

And how do they judge that? ROI, right? How— what are the numbers that go into kind of that calculation?

It depends also on the type of customer that you are selling. And let me tell you what I mean by that, of the business unit, because there are 2 ways to have— 2 straightforward ways to have ROI. One, cost-based. Every transaction that I have here that is being performed by humans costs me this amount based on hour plus number of transactions. And now with AI, it is this cost. So I can now allocate that human capacity to keep doing special things on that or other things on the company. So it is a cost, literally simple ROI, no?

Mm-hmm.

Which, by the way, if you go with frontier models, good luck. If it is high volume and you have 10,000 of these, you are not going to have ROI.

Mm-hmm.

On the other way, the second way of how you can get the ROI, it's based on one of the core values that we have for these regulated industries, that is the acceleration of the delivery of value to their clients. What it means, I want to pay, I have an approval for, to pay, for example, I need to do the proof of income and it takes me 4 days to do it because today I have a team of humans involved. It's not about cost. This is a high-volume business, but now I'm able to do it in 1 hour. In that case, I can market that to the dealers. I will pay you in 1 hour instead of in 4 days. What do you think that happens to my market? Do I get more market by being able to answer to them immediately? Or, or so it is not— the ROI doesn't come because of the cost reduction.

That also.

It comes also because of the revenue increase, because I have— I directly provide more value to the clients, which lets me market it, which increases my market size, for example, over that business. This is just one example, but this is how they mostly see it. And to be honest, 99% is focused on the cost.

To what extent are you replacing robotic process automation from the last generation, like UiPath or OCR kind of tools? Are you displacing some of those tools when you deploy these agents on-premises?

To the extent I tell you, to this extent, we are doing what they cannot do and will never ever be able to do, which are all of these knowledge of tasks that are core critical in the business sites. At the same time, we are going backwards to reevaluate if having a mainframe of IBM of 1998, it's still the best option, or I can just connect to a system of records, remove that from the table, stop paging these robots to with the UIs and just be able to transition without paying the cost of transition. So yes, we disrupt significantly the RPA market because we can do what they do, but what is more important, we can do what they cannot do because of their literal infrastructure.

And does that involve a lot of computer use? Are your agents basically doing computer use or are they— Interacting via API with the systems?

A lot of these systems doesn't have APIs because they are legacies and you have to directly access systems of records to manage them, or they are old mainframes and you have to go to very, very old kind of databases, which we eventually usually advocate for a creation of an MCP, a way for them to expose Because this is also a decision of our clients. Do you want to keep using legacy systems or do you want also, while you update to AI, to rethink the processes and make them much smoother with the tech? But we don't use computers in that sense. We don't click on buttons. We use a completely different approach that Manu will explain that runs and creates the code in real time that will make me navigate that iframe that changes Literally on every execution that I do of SAP or Salesforce, for example.

Right. So your digital workers actually end up using the SAP or using the Salesforce that—

Yes.

I see.

Yes, exactly.

I see. And via MCP tools or something like that?

Or interacting with them from a browser perspective, but without using vision because it's very costly, slow, and Just writing the code, like using the DOM object of the website, or there are a lot of ways of solving that.

One question I have for you is that, does that break some of the terms of service of these softwares? Because Salesforce is like a per-seat software and, you know, are they allowing you to do it or, you know, is the user just kind of allocating his login or how does that work?

No, you create a user for the agent. It's just a user that writes, that uses the keyboard very fast.

SaaSpocalypse now. In the last couple minutes, I'd love to just change gears and zoom out a little bit. Big question for me, you guys are in Spain. I saw on the website you have 3 offices in Spain, 1 in the Bay Area. And, you know, I live in Detroit, Michigan. My perception is that the Bay Area is sort of uniquely AGI-pilled. Everybody there is running extremely hard and the rest of the world is all different. What is AI culture like in Spain? Are people AGI-pilled? Are they doing 997? Uh, are people still taking siestas? Like, what is the, what's the report from the cultural frontier of— I lead understanding and working with AI in Spain?

The individual that has the most open source models in Hugging Face is in this call. He has 700 and 180 million downloads and he's from Spain and from the west. So, and he's leading the AI team here. So how is the culture here, man? What do you think? What is your opinion?

No, thank you, David. Yeah. Well, in Spain, in Spain, as in many other countries, you have people that who like, who like taking naps, and other people that, as me, we— I had no social life, no girlfriend. I just started creating models and more models. I just started collaborating with Hugging Face. Yeah, it's a tricky question. We were like 2 weeks ago, no, David, we were in San Francisco, and yeah, they are super AI people, but you The world is so globalized, you can do whatever you want in whatever place in the world. I think in Spain, especially in young people, I'm seeing that they are like very— they are receiving AI with a lot of energy of doing something different. But yeah, it's like we know in San Francisco everything moves so fast. And yeah, it's different. We want to go there quite often because we can check how we are, how if we think in the right way, how they are doing. But yeah, even if we are in Spain, I think the most important thing is that we share this passion for AI. We read a lot of papers, we are in touch with people in foundational labs and so on. If you love something, you can develop even foundational AI or this kind of solution wherever in the planet, as little as I see it. Yeah. I think in San Francisco, maybe someone take an app also. I don't know.

Yeah.

I had a viral post on that. And I will end up with this. The problem, the good thing of building here and that we see adding to what Manu said, It is not about working 100 hours a week. That can work running release dates and so on, and that culture that is growing in this area, but it will absolutely kill you. I think that what we have here is that on Sundays we have sun, extremely good food, good weather, and an extremely good retention of talent because who comes here doesn't want to leave to other place. So I also value that. And I think that we are extremely lucky.

Balancing how exactly to sprint through the AI marathon is definitely a challenge that we're— I personally am thinking a lot about these days as well. I don't know, Prakash, if our next guest is not yet here, do we have time for one more question or are we ready to make a move? Looks like we might actually be ready.

Snehal just came into the green room, but we have time for one more question. Just go ahead.

Okay, cool. I was also gonna ask David, I don't know if I should be saying David, but you have the interesting background of being a magician before getting into AI. And something that I'm really interested in from a big picture perspective of like, where's all this going is AI deception. So you're, as a magician, sort of a master of deception. And I would be very curious in your take on AI deception. Do you see examples of deception? Do you think they are any good at it? Do you see— how do you feel like the trajectory of their ability to deceive compares to their, you know, broader trajectory of capabilities advances? I think you probably have a pretty unique take on the problem of AI deception.

It is quite unique. There is one challenge that I said, Manu, for a long time. I would love to have an AI that is a magician. I don't know if it's possible. As you know, I know how it works. So I've been personally also trying to say, can an AI make you a magic trick? Because for me, it would be my Turing or my AGI test is that one. The day that the AI can make me and execute me a magic trick, I will tell you that AGI is here. But— We will see. I tell you my perspective on deception. It's that the problem, it's related to maybe the AI psychosis. I don't know if you have heard that term recently, but I think that is probably the most interesting topic on AI. And I think that it's extremely well at doing it without it being conscious. We expect, I expect of it doing it. What I mean by that is that My take on it is that the problem that we don't see is that AI, everyone is thinking on LLMs, but AI since the last 5, not 5, 10 years is dictating its own path. The content that you see is content that is done by an algorithm that you don't control, which is AI. The content that gets recommended to you, it's content that you don't control. And all of those forge your mindset and your perception of what it reality and your ideas, wanted it or not. And then the one that tells you that thing also was influenced by another AI system that might know because of graphs that you are connected. And if you think farther in the future, these systems are for me like mycelium. Like mycelium. I don't know if you know mycelium, if you know what it is, but deception can be played in multiple levels. on the stages, and that's how magicians do it. The trick is not done when you think it's done. Usually it's done way before you even realize that I'm doing to you a trick. And, um, that's something that I'm very cautious of doing it, that we are already having AI psychosis. For me, it's a quite red flag on a few things, like, wait, um, what if this chest keeps growing and, and getting embedded into our digital environment, um, because very soon we will absolutely lose control, which I don't think it's just from a, from a stake perspective. It is what it is. So let's see how it goes now. But I, I always like to— I've, I've gone that rabbit hole and it ends up saying I think that it's already manipulating us, to be honest, like in ways that we don't understand yet. Uh, it's simply But it will take a bit of time. Again, because as a magician, the trick is not done when you think that it's done. The tricks happen whenever you realize it and they just go and happen.

Nice little flourish there. That's a sobering note to end it on, but it certainly does reinforce the need for all sorts of reliability and control paradigms like the ones that you're developing in the enterprise. It's an interesting confluence of your background and your current mission. We'll definitely be— if you start to see anything that, you know, you feel crosses important thresholds, give us a shout because I definitely want to make sure I'm on top of the latest in AI deception for my own and everyone around me's benefit. But thank you guys for joining us today. This has been an interesting conversation.

Thank you.

Thank you so much.

Thank you.

Awesome. So I will jump directly to our next guest. Snehal Antony is the CEO and co-founder of Horizon3 AI, the company behind NodeZero, a platform that runs automated security tests against real company systems. The idea is to move cybersecurity from theory to proof Instead of producing a long list of problems, Horizon 3 tries to show which weaknesses can actually be combined into a real attack path. Anthony's background is unusually relevant here. He has held senior technology roles in the US Special Operations. He was in JSOC, Splunk, and GE Capital. So he's seen security from both the military software and large enterprise sides. His current view is that AI will make cyberattacks faster, but that defenders should not blindly hand control to AI. The interesting tension in his work is how to use AI for reasoning and prioritization while keeping the dangerous actions constrained. He's going to be a very, very interesting guest for us, and let me bring him up on stage. So, hi, Snehal.

Hey, awesome to catch the last part of the previous conversation. I'm excited Excited to dig in.

Snehal, most companies think of cybersecurity as a checklist. Patch the known bugs, buy the defensive tools, close the tickets, hope the system is safe. But the attackers don't experience the company as a checklist. They experience it as a maze. They try to grab a password here, a misconfigured system there, a forgotten machine. I think the most classic one is the S3 bucket, which has not been closed off and is open to everyone. I think that's leaked. I think at one point that leaked all of China's ID card numbers, like 700 million people or something, were leaked in one go over an unsecured S3 bucket. And they combine these small things into becoming a larger attack. Your work at Horizon3, you claim you don't just ask whether they are You know, some weaknesses, but you ask whether an attacker can chain them together to prove real damage is possible. So when Node Zero tests a real company, what does it reveal that the security team genuinely did not understand about its own environment?

Yeah, so a bunch of interesting thoughts or questions there. One, can you hear me? You guys hear me okay?

Yeah.

All right, cool. So first and foremost, attackers don't hack in using zero days like you see in the movies. Usually they're just logging in with credentials that they've compromised through a variety of techniques. And so one of the more interesting attack paths is we find a way— NodeZero finds a way to compromise a domain user, whether it's through password spray or exploiting Veeam Backup and Recovery and pulling creds from memory, whatever else there might be, a variety of ways to compromise the domain user. With that domain user, you can log into a host over SMB in a way that completely bypasses any of the signature-based detections of an EDR. In fact, the leading EDRs only catch that particular attack path 16% of the time— 1/6. So now you've logged into the host. From there, you can dump SAM, dump LSASS, dump other sensitive processes, iterate, iterate, iterate, and eventually become domain admin. So I think what's been amazing in We now have run 250,000, I think, pen tests total. What's amazing throughout all of them is it's not some crazy obtuse zero-day. It's usually not even a CVE that's required. It's credential compromise chained with misconfigured EDR combined with some other misconfiguration, whether it's in your data security stack or something else, that leads to an effect. whether it's sensitive data exposure, domain admin compromise, and so on and so forth. And so you'll always catch these like crazy stories of the one S3 bucket that led to 700 million credentials breached and so on and so forth. But man, those are outliers to some degree. It is the daily grind of an attacker spending their time deeply understanding your network, finding those edge cases in misconfigurations, compromising credentials, chaining things together. Usually misconfig, dangerous product defaults, rarely a CVE. I think that's what we've seen across the board. And to my surprise, very— even for me when I was a CIO, I didn't really understand the anatomy of an attack until I got in to deeply understand these attack paths. And so credential compromising or credential attack surface being the most important to get after is something that I've only built more conviction around. Does that make sense?

Yeah. So I wonder, you know, you have experience both on the military side and I think I would call it in private practice. Where do you think the big differences are in how the organization approaches these issues?

So the But you know, one of the things, misconceptions of special operations, and remember my military experience was watching Jack Ryan in Tropic Thunder. Like I'm not a soldier. I wasn't a shooter. I had the privilege of leaving industry to work inside Joint Special Operations Command, which is one of, it was almost a PhD program in leadership, to be honest with you, being able to work alongside the greatest leaders the world will never know about. But what made these folks special? Is not some crazy mission impossible maneuver dangling from the ceiling. They mastered the fundamentals of shoot, move, and communicate, and they trained like they fought. They built the muscle memory, so when they were in crisis, they were they already they knew what to do. They were it was second nature to them. So probably the fundamental difference in what I see in special operations, and we apply that to cyber, we apply that to all aspects of. Of, of that world is training like you fight from a cyber standpoint. If you're in a breach, you don't want to learn how to respond to that breach in the middle of it.

Yeah.

You want to build that muscle memory ahead of time. You want to master the equivalent of shoot, move, and communicate, uh, so that when things do go wrong, you know exactly who has the authority to make the decision to kill a connection, who has the authority to shut the entire network down because that's the safest thing to do. Knowing there's risk to the business or revenue impact. You want to know exactly what the timelines are going to be, and the attackers are only getting faster. And so, you want to make sure you understand what is the clock starts ticking and how quickly must you make a decision to prevent the situation from getting worse. So, I think the fundamental difference from a cyber standpoint between my experience in special operations and my experience in the industry was this desire to train like you fight as a core part of what you do every day that most organizations honestly don't do. Those that do it well tend to also take disaster recovery seriously. So in financial services, you force your data center to shut down every quarter to practice disaster recovery, to pack— to practice data backup and recovery. If you're not practicing, you don't know what's going to happen when things actually go wrong.

Security teams already know that they have thousands of vulnerabilities. I mean, like the average, like, you know, hospital, probably, you know, they have all these HIPAA things, which everyone's supposed to follow, but, you know, you can basically just walk through the parking lot and like listen in or walk, you know, just take a walk through the office. Everyone's written down their passwords everywhere. Just take a couple of photographs. Jander walking through probably, you know, can get admin access fairly easily without too much, you know, trouble, right? So how do you make the case that, you know, these companies already know that they have, you know, vulnerabilities, they have tons of them, and most of them still don't, like, you know, bother to have, sufficient hygiene? You know, YubiKeys, for example, YubiKeys should be more widely used, but I have not seen any hospital I know use YubiKeys for logins, for example, because, you know, username, password, and that's it, right? So how do you help these? Like, you can do the pen testing. The vulnerabilities are there. You can change the attacks together. They know them. But how do you How do you move these organizations into the next step of actually addressing them?

Yeah. So when I was a CIO, one of the hardest parts of my job was deciding what not to fix because I had way too many vulnerabilities. And so how do you think about prioritization? Number one, is the issue even exploitable? Let's just start with that. Just because you had the Log4j JAR file doesn't mean you were exploitable to Log4Shell. You could have put an RMI outbound firewall rule in place that kills the entire egress and— or as an egress rule and kills the entire kill chain. And just because you've got the JAR file doesn't mean you're exploitable. You've put in a mitigating control or compensating control. So step 1 is, is that vulnerability or that issue even exploitable in my network?

Mm-hmm.

Number 2 is, are threat actors known to use it? And number 3, most importantly, is what's the consequence of exploitation? Because the second hardest part of my job was looking an IT admin in the eyes and telling him or her they've got to skip their kid's basketball game tonight, stay back and patch a server, knowing full well that that had nothing to do with cyber risk because it wasn't exploitable. It wasn't known to be used by threat actors, and it had no consequence to the business. So let's just take that as the premise. It doesn't matter what tech trend we're in— AI, not AI, so on and so forth. What's important is answering those 3 questions to prioritize. The next big issue, though, is remediate. Remediation is the bottleneck. No matter what era we were in, we didn't have enough IT admins and network engineers that knew how to fix stuff. So how do you accelerate remediation? When I was at Splunk, we acquired Phantom. And one of the challenges in the SOAR market— that actually didn't really work out as a great acquisition because the Venn diagram of IT admins that knew how to write Python was pretty small. Now with coding agents, you actually create the opportunity for IT admins and network engineers to actually build automation and scripts and so on to help remediate at least the things they see over and over and over again. And so I do think that we're going to see this new era of automated remediation and automated mitigation where if I can't patch a server fast enough or fix a code flaw fast enough, at least I can put a band-aid in through a firewall rule, or at least I could put in a more aggressive EDR rule or a DSPM rule or whatever else I've got to go do to buy me time or to minimize the blast radius. So I do think that prioritize what matters and then set conditions to quickly fix Things that matter are the most important because the goal of running a pen test isn't to find problems. It's to quickly fix problems that matter, which has been a philosophical difference between those that were truly defensible and those that check the box from a cyber standpoint.

So do you think that that mix changes? I mean, all over your website and all over every part of the cybersecurity world right now, is mythos, mythos, mythos, right? And I'm sure that brings up questions for you at basically every level of analysis. One is like, maybe while humans were often the weak link, maybe we're gonna be in a period now where those zero days like we see in the movies really are the thing that gets opened up and it's a, it's a totally crazy period of time. Um, another possibility, and I know you do some work in the intersection of formal methods and cybersecurity, is maybe we can actually get the AIs to develop provably safe systems, and then we can actually take our siestas again because we're confident in that. I am very confused. And then another big question, obviously, for you as an entrepreneur who's raised $100 million last year, which used to be a real war chest. Now we've got a world where each lab is spending 10 to 100 times that reportedly on just reinforcement learning data, of which some non-trivial part is going to be clearly related to cyber. So how have you had to change your strategy, and how do you think the balance of power is going to to shake out in light of what seems to be pretty critical thresholds crossed?

Yeah. So I was really lucky when I was at Department of Defense to be involved with Project Maven early in the day. So Maven was the AI warfare task force or project applying AI to combat. This was 2018. So I got exposure to basically early DeepMind, what became OpenAI, so on and forth. And one of the things we learned pretty early is that the models themselves are disposable. They're changing so often, you're just going to throw them away every 6 months, every 9 months. The 2 parts of the stack that are truly durable are the harness and the training data. And those are the things that you need to get right. So why does the harness matter? The harness is the difference between being production safe and not safe, right? So number one, And then the training data is super important because in cyber in particular, attackers live in— sorry, attackers live in the edge cases and LLMs live in the mean. And you've got to really, really take that into account. So why is that interesting? Mythos. So think of the training data that these frontier labs have access to. Anything regarding software, like actual code analysis, the labs are going to kick everyone's— like, they're just going to crush everybody because the cost of training data acquisition is basically $0. The 3 of us could go start an AI company right now, go start a web app pen testing company, and build agents trained on every Git project, every Linux Foundation project, and every merger request. There is no barrier to entry for training data, which is why anything source code analysis related is going to be a huge advantage for the Frontier Labs. So what does that mean for cyber? The effort to find, to do vulnerability research is basically going to zero. And so that's why we're seeing tons of code flaws being exposed. Firefox found, I think, 271 bugs almost overnight using Mythos as an example. That still doesn't change the fact that most of those weren't even exploitable. Like, all right, you found flaws, cool, but it weren't even exploitable in your environment. But where these models are actually struggling, if you double-click on the data for ethos, it actually regressed compared to 4.6 in runtime exploitation. The reason why is, last I checked, JP Morgan didn't publish their network configurations online anywhere, or their Active Directory configs, or their data security configs. All of the most valuable data in cyber is behind the firewall. All of those configurations, all of the edge cases that are there, the labs have no access So what we're actually seeing is this bifurcation between source code analysis, which is really great, and then actual runtime capability. Number 2 is because these models were trained on extremely limited training data. And the analogy is at Maven, we were really worried that the adversary would corrupt our training data to make an aircraft carrier group look like a flock of birds.

Right.

That's what we were worried about. And there are examples of this, like people using facial recognition at protests and the protesters defeating them with patterns on their face. It's a form of adversarial AI. When you train on extremely narrow, shallow data, which is what the labs have trained on— hack the box ranges, capture the flags, and so on— we've actually found these models to be extremely gullible and super— and you talk about deception earlier— super easy to deceive. And so the really big question is, can these models actually withstand an environment that has the enemy within it, right? And a network that's actively defending with basic decoys and deception. And all of our research has proved, so 4.7 is gullible. It's deceived by simple basic deception 56% of the time.

Hmm.

4.8, 38% of the time. Those are the latest models, and that's basic deception, let alone a web of lies designed to, to get these models to burn tokens, blow out their context windows, or chase dead ends. And so I do think that these things look really good in cyber ranges, but in networks where the enemy actually has a vote, you're not seeing them come to fruition because they've got a fundamental flaw in the diversity, and depth of the training data they built their decisions on, because there just isn't enough infrastructure training data out there, um, compared to the app side of the house where they're always going to be the best. Does that make sense?

Yeah. Do you think there is a prospect for using something like formal methods to make that firewall robust enough up front that those sort You know, once the mouse is in the house, I take your point that it's like AIs are still not that adversarially robust. They are relatively easy to trick. The trend is down, but, you know, it's not down as fast as other things seem to be up. But the hope that I am kind of getting from a bunch of people is maybe we can close all these vulnerabilities, shut everything down, you know, and get to the point where We don't have to play that game anymore at all. Does that seem just totally fanciful to you, or would you put some stock in that?

Yeah, I mean, there's plenty of evidence to show that vulnerabilities are rarely part of a cyberattack, right? 0.5% of vulnerabilities become CISA KEVs. Even then, the CISA KEVs tend to be in 2 layers, the perimeter gateway and remote management tools. That's it, right? That is where CVEs matter the most. The rest of those attacks, it's living off the land, it's credential harvesting, it's misconfigurations, it's things that have nothing to do with CVEs. And so I do think that everyone always looks for that easy button to press and hope that everything goes away, but that's not how attackers operate. They've never operated that way. As vuln scanners became better, they pivoted. They moved away from CVEs towards misconfigs and credentials. And as we start to defend misconfigs better, they're just going to move to the next thing and the next thing. It's like water. They always find a path. And misconfigurations and credential compromise in particular are 2 of the most straightforward ways that LLMs and AI agents have very little effect on.

So, So I have a question. Mythos was obviously like very hyped, right? Very, very hyped. And post-Mythos, there were disclosures from OpenAI that GPT-5.5 could have found those bugs and just no one bothered to try, et cetera. There was also this sense that Mythos was not Like, uh, published widely because, you know, Anthropic just didn't have the servers, the compute necessary to, to allow people to use it. And so they ended up with this kind of like marketing spiel that it's too dangerous to release, which we've seen multiple times before. Um, it also created the sense of like the haves and the have-nots because they had maybe 15 or 20 organizations that they started out with. And other people in the security community were not actually offered access. What did you feel in the security community about that? Like, did you feel people were like, okay, there's now a first tier and second tier, and the first tier guys are going to get the bugs first, and the second tier guys are just going to have to wait? Is that, is that that perception of the market kind of bifurcating?

I think you've got to split the market up into— so the positive of Mythos was it scared the hell out of every board out there and it raised tremendous awareness and it gave the CIOs, the CISOs, and the practitioners the budget and the air cover to do what they've always been saying they needed to go off and do. So it was a huge positive in that regard. The other part though is there's plenty of research and plenty of activities in the practitioner area and so on that show you didn't need Mythos to do any of this stuff, right? Using LLMs for vulnerability research has been going on for 2 years.

Mm-hmm.

And there's plenty of papers that came out afterwards that showed all the problems that Mythos found, anyone could have found with any other model and, you know, the same harness or even a slightly crappier harness, to be honest with you. So I think they did a wonderful job at scaring the market and giving everyone budgets. Good job, right? Congratulations. Outside of that, I think there was a little bit of the have and the have-nots, but that was more marketing. Like, oh, you must not be an innovative company because you weren't in the original list of Glasswing participants. But I think that dissipated pretty quickly. There was a little bit of that at the beginning, I think I saw. And so I think what's happened is, and this is going to happen again and again every 6 months, There's gonna be another Mythos-like model that gets released and all the stocks are gonna crash. So, you know, get your Claude bot to short a bunch of these stocks as quickly as you can and then buy the dip, except it's always dipping. Um, and, uh, and go from there. However, I just, I think it's done a really good job of creating budget, but at the practitioner level, there's definitely a, a collective eye roll around what was real and what wasn't real.

Can we go back to the human side a little bit? As I, you know, confront the problem that I'm probably the weak link in my own system, what do you think that solution looks like? You know, what would you recommend to me today? Right? I've got, like, agents running all over the place. I'm getting more into the habit of having them assess security. I honestly never used to care. But these days I'm like, I guess I should take this at least somewhat seriously. So I'm prompting Claude to tell me what I'm missing, doing wrong. But I'm still getting like, I am starting to get actually increasingly sophisticated spear phishing sort of attacks. And I don't think I've fallen for any yet that I know of, but how do I harden my own cognition so as to be more Robust and not ultimately be the human vulnerability that undoes myself.

Yeah. I remember when I was a CIO, one of the most effective phishing campaigns we ran was sending out a message that said, we detected you accessed adult material, adult websites from your work laptop. Click here to explain why. It was like a 90-something percent click rate on this, and you had to log in in order to get to the freeform text field to explain why. So now that you've logged in, we've got your user ID password. And then the freeform text, by the way, was absolutely hilarious. I remember one person wrote, I'm usually very careful. I don't understand what happened. And clearly incognito mode didn't work for that individual. But anyways, the point is this. A motivated attacker is going to find a way in. It's just inevitable. And it's only easier now with deepfakes and less misspellings in your spear phishing and so on and so forth. And what you really just have to do as a CIO or CISO is assume breach. It's never been more important to not only assume breach, but we're in the blast radius management business. You assume they're gonna get in. All you've gotta do now is focus on how to minimize the damage as effectively as possible because there is no way to stop that initial access. And I think people that believe they can stop it altogether are— it's just impossible to do. There's way too many doors, way too many windows, way too much of a human element. Even if you're really good. So here's an example attack path. Even if you're really good, at network defense and so on. If I can log on to a server as a compromised domain user and the EDR isn't configured right— and we see this quite often— now I've got host compromise. I can do things like pull the MFA auth token out of Slack's desktop application or out of Office 365 and use that MFA auth token to access the Slack workspace as that user. or access the Entra configs as that user. And that's all post-MFA. That's post everything you've put in place to prevent that credential from being compromised. And now as a Slack workspace user, I can do whatever I want.

Mm-hmm.

And that's actually how Disney got hacked a couple of years ago. And so the human element is only going to be more exploitable. And so you have to assume breach and do everything you can to focus on being defensible versus just being secure. Secure is a point-in-time state. Defensible is the enemy's getting in, and how do you adapt as quickly as you can to keep the enemy from achieving their objective?

So one note is that I have personally done that MFA thing for my OpenClaw because in order for your OpenClaw to act in certain circumstances, it's going to need your auth tokens. And so my OpenClaw has definitely extracted my own auth tokens in order to act on certain platforms.

Yeah.

So, how do you square this? Because, you know, I see Jason Calacanis, you know, he's like, I have 30 OpenClaws, and like, they're all over the company. And obviously, they have access to the network itself, and they're operating within that network. How do you square— and, you know, you talk to a CISO, and they're like, no, you should never do this, right? But obviously, these executives are doing it anyway. How do you square that? Like, where is this going right now?

Part of it comes back to risk. Part of that comes into what's the best way to get more Twitter followers. Part of that's how do you make yourself look like you're at the bleeding edge? At the end of the day, an agent, whether it's an open call agent or whatever else, is what? It's a new form of insider threat risk. So, when I was a CIO and you worried about insider threat, what did you worry about? out, a user that is a target or has bad hygiene, that's over-permissioned, and as a result, poses some threat or risk to the organization. So what's an OpenClaw agent? It's easily maligned. With basic effort, you can get it to hate the world. It is way over-permissioned, right?

Mm-hmm.

So agents are just the next generation of insider threat, which has always been really hard to If you're in a bank and you've got a really strong insider threat program, to extend that to cover agents is fairly straightforward. If you had nothing in place, or if you were already weak at insider threat and you start to deploy these agents, you're generally screwed.

Can we zoom out and assess what that means for international competition as well briefly? Obviously, we've got got a lot of different motivations pushing people to accelerate the frontier. But one big one that we have heard from no less than Anthropic CEO is that we want to get ahead. We have a lead, we gotta keep our lead vis-à-vis China, then we can maybe get to some decisive strategic advantage, and then we can make 'em a deal they can't refuse or whatever. I'm not sure. I think they've softened on that plan maybe a little bit. But If I try to jump to, you know, try to port a lot of what you've said over to that context, it would suggest to me that you're not a believer in that strategy and that you think we're probably going to have a much messier and more human rivalry between countries for, you know, at least the foreseeable future.

I think so. I make a face because You definitely have to take a step back when you start to get into strategic competition. Uh, the tech is a part of it. It's, it's much more than just the tech. So, um, what does, what does an AI attacker bring to the table? I grew up playing, um, Nintendo and Contra, and the moment you, you know, without the cheat code, you had limited ammunition and you were very selective of the targets you were going to go off and shoot. But what happened when you put in up, up, down, down, left, right, left, right, B, A, What happened when you put in the cheat code? Unlimited ammo, full auto, indiscriminate, no selection of targets, crush everything you could. AI cyber weapons are basically unlimited cyber bullets, right? You no longer have to be selective of the targets you're going after. That's one massive shift. The next massive shift is there's no longer a distinction between military and civilian targets. We saw this with Iran. AWS data centers are now being targeted. There's no longer that distinction. And when you've got those 2 things in place, strategic competition and gray zone warfare look totally different. So for example, back in 2022, the Japanese provided support to the Ukrainians. The Russians got angry and ransomwared a small company in Tokyo, and that company in Tokyo provided cup holders to Toyota. And due to just-in-time logistics and lean manufacturing, Toyota shut down 28 production lines and cost them almost $400 million of economic harm over cup holders. The flex by the Russians was not that they could ransomware a small company, it's that they knew where to apply the least amount of effort to cause the maximum amount of harm, economic harm, below the threshold for war, right? That's the game. AI is an enabler for But it's all about synchronizing effects to cause economic or political turmoil to achieve an objective. So if I use my unlimited cyber bullets, if I'm the Chinese or whomever, an adversary, and I go after an oil refinery, the Colonial Pipeline, and a ship transiting the Suez Canal at the same time to dramatically affect oil supply, and oil prices spike to $100 a gallon, gas spikes $100 a gallon 3 days before the election in November, Cyber played a role, but it was how you use that combined with other effects to achieve an objective. That's what matters. So when I hear that, it is a little bit of making AI the center of the universe. That's not really it. It is an enabler towards this broader effort and trend we've already seen for the last 5 years around cyber-enabled economic warfare below the threshold And the last example I'll give is last year there was a huge spike in cyberattacks against rare earth element mining companies in Africa aligned to the West, in particular the HVAC systems and ventilation systems of these mining companies. Why? Because if you can cause tragedy and cause loss of life and reduce capacity of rare earth element mines supplying the West, that's a win for Russia. Well, you're going to be able to do all sorts of things to drive more demand for Chinese rare earth elements. So I think that you think of AI and the element of cyber-enabled economic warfare, strategic competition, and I think that is the framing in which to use. And then models and distillation models are a step in that, but it's all the other pieces that come together that make it relevant geopolitically. Does that make sense?

Yeah, I don't necessarily feel like that plays to our strength, our being the West's position there.

I mean, we operate with rules, we operate with morals, we operate— we have a distinction between military and civilian targets, right? We have that intentionally to an extreme. The adversaries don't. They use proxies. They go over and do these things. And so, you know, I think that if we have an advantage, it's because we don't— we maintain that distinction between military and civilian. We aren't as overt about these things. And, you know, should that change?

Maybe.

You're starting to see more liberal— or you're starting to see policy changes at the White House with the national cyber director and others. on the use of offensive cyber and hack back and so on to level the playing field. But we definitely are operating with our handcuffs and we're operating constrained compared to the adversary in the realm of cyber today.

Fascinating.

Indeed. I often wonder what cybersecurity will look like in 2 to 3 years. Because now that the LLM capabilities are kind of known, agentic capabilities are kind of known, you can kind of predict that they're going to get a little bit better and they're going to be human-like within 2 to 3 years. How does that change your positioning or the positioning of, you know, how the enterprise should react and how, you know, organizations should react to near-human capabilities digitally, at least?

Yep. Um, our AI hacker Node Zero— when we pioneered the whole concept of AI hackers— hacked a bank in 77 seconds. No humans involved. Point, click, shoot, go. And that bank had every Gucci security tool you could buy. 77 seconds. If you can't stop us in 76 seconds, it's game over, because that second 77 We get domain admin. We're locking you out of the building. We're locking you out of your machines. We're rampaging and ripping through every aspect of your environment.

Hmm.

So think of 77, 76 seconds. Can your defenses actually stop us? Can you move that quick? If you doom scroll for a brief moment, 76 seconds has passed. If you go to the restroom, if you get a coffee, it's passed. There is no way a human in the loop is going to react So the future of cyber warfare is AI fighting AI with humans by exception. It's the 77-second problem, and that's only getting faster. Faster. 2 years ago, we had a 9-year-old kid hack a bank in 4 minutes and 12 seconds. So in 2 years, we went from 4 minutes and 12 down to 77 seconds. Probably we're starting to reach like a theoretical limit, to be honest.

Mm-hmm.

And let's just say that theoretical limit is 30 seconds. Can you move that quickly? The other part is our AI hacker makes, I don't know, let's just say 1,000 decisions a minute at the moment.

Mm-hmm.

Pretty quickly, 10,000 decisions a minute, 100,000 decisions a minute. There is simply no way a defender can keep up with that. Now you'll say, and we're starting to see today, oh, I'm gonna have a human manage a bunch of defensive agents. And when I was at Splunk and when I was at GE, a really good SOC analyst could do 20 alert investigations a day. A great one, a senior, could do 30 and a junior could do 10. So human limits are 10 to 30 alert investigations per day. Even if you used an Iron Man suit of AI agents, maybe you get to what, 100 investigations a day, 200 investigations a day. If I'm generating 1,000 critical alerts a minute, there's simply no way to keep up. So what I believe cyber looks like in 3 years is unfortunately tragic. It's gonna be tragedy first. It is gonna be way easier for the attacker to move at 77-second speed than it is for SOCs to rebuild everything to be humans out of the loop. And so I think there's going to be A massive dramatic change. I think every single incumbent SOC tool right now is up for disruption. Every one of them was designed with either humans in the loop or humans on the loop. They have to be rebuilt to be humans out of the loop. And it's going to take— the attackers will get there way faster than the defenders will. And that's what I think it looks like.

Incredible.

That might be a sobering note to end on.

Not to be Dr. Doom, but like, it's just the reality of the math. I mean, you know, you can work backwards on those things. And, uh, the dis— we saw this in fighter jets. Like, the moment you can— your OODA loop, you're inside the enemy's OODA loop, you win, right? The attacker is going to be in our defensive OODA loop. They're already starting to get there. It's only going to get worse. And because of bureaucracy and technology and sales cycles and all the other things that happen, It'll take extremely long to rebuild the entire defensive stacks to be humans out of the loop.

On that note—

No time like the present to get started, I guess.

Yeah.

We'll let you get back to work.

Thank you. Thank you for joining us. It's been a pleasure. We have learned so much, and we definitely hope to have you back, and we definitely hope that your predictions do not come true.

Well, I'll end with this, and it's a great tie to your previous guest. These models are inherently gullible. Deception is actually, I think, the EMP of AI cyber weapons. If you have really good deception and you honestly, you throw honey tokens everywhere, because these things are immediately, these AI agents are going to start living off the land. The moment they live off the land, they're going to look for things they've seen before. If you just put the juice shop web app on your network and with a bunch of sensors, these AI agents have a bias towards chasing. So the mitigation between now and then is deception. I've become a huge believer that decoys and deception are going to level the playing field for AI-based attackers.

The honeypot.

Yeah, make deception great again.

Make deception great again. Awesome, awesome. Thank you. Thank you, Sneha.

See ya.

See ya. Wow, that was, that was sobering.

Yeah, it's, it's very interesting. I mean, I'm out of my depth here. This is a great example of where it's just getting really hard to track, you know, to chase these things down to ground truth. And at the end, when he mentioned the fighter jet thing, I did an episode of The Cognitive Revolution not too long with a couple of legends in the formal verification space. And the way that they presented those old experiments from the military was like, we were able to effectively use formal methods to wall off certain critical systems well enough that, yes, if they got in, we had a huge problem, but we were in fact able to keep them out. You know, certain assumptions there, I suppose, around human vulnerabilities being not— I mean, that wasn't really the thing that they were testing, right? But they did put an attacker like on one system within a helicopter, as I recall it was, and said, you know, can you break from one system to the other system? You're on some shared hardware. You know, there are in theory, you know, ways that you would get through, but because the defenses were so hardened, they went from, in the early experiments, it was like very, you know, it was one of these 77 seconds things to later on, like they were actually able to fly the plane or fly the helicopter without any issues. And I just don't know what to make. And this is like a recurring theme, I guess, in AI in general, that you have these like divergent opinions even among world-leading experts and To some degree, everybody's kind of talking their book, but I also think it's like more, much more sincere than that. I don't think the cynical read is the right read. I think people in many cases like have the book that they have because of the opinions they have.

Definitely.

So I don't really know what to make of it. And I just find myself like totally confused as to how should I bet right between his vision of it's going to be disaster first versus the AWS and DARPA vision of we'll formally verify everything and we'll lock things down. And that way we can sort of, with the AI help, we can code our way out of this problem.

I don't know.

It's really confusing. How do you think about even making sense of that? Or I guess I guess to some extent we have no choice but to live in the mystery, but I can't say I'm content with that.

So I feel like people and systems evolve to— systems evolve to where people want to be. So obviously right now everyone wants to figure out how to use this OpenCLaw thing, right? And how to use agents to manage their life, to manage their digital life, to take care of their emails, to answer Slack messages. And the moment you start seeing it like work and it saves you an hour or half an hour, you're like, okay, now how do I hand off more and more to it? Right. And it's these products, you know, my friend always says, you know, the product gets pulled out by the market, right? The market is there and it pulls. And this is actually, you know, that guy Rubin, like the David Rubin, Rick Rubin, the entrepreneur who like helps musicians like create stuff. Stuff. He also says this, like, it— like, it— the, the atom— the environment pulls it out of you, right? And so the product gets pulled out by the environment. So the market is now saying people want to use these OpenCLaws to manage their things. They want their OpenCLaws or their agents to go into Slack and extract thousands of messages and, you know, have a history. And people who fight Like, you know, for, you know, some of these systems of record, Slack, for example, it wants to constrain the number of messages you can pull out of Slack, right? It wants to charge you on a per-message basis, because if you're a user, if you're a human user, you use it just like, you know, you see like 10 messages. If you're an agent, like the Mysa guy, he says, you know, that their agents are typing very quickly, so to speak, right? So what's happening, what'll end up happening is that the systems will get created to— the products will get created to allow the users to use these systems as they want to use them. And people who do not create those products will lose out and users will move on to other products. And that's a fact of the matter. And everyone in the system, whether it's you're providing cybersecurity or bank software or whatever, will just have to live with it and transform. And organizations that don't will be stuck, right? I believe to some extent it's already happening in banking. And banking, I think what happened is like the regulators enforced this kind of like product surface, and you ended up with crypto, which was above that product surface, which basically kind of took out the non-KYC transactions and international transactions and And all of these other transactions that could have been happening within the banking system got pulled out of crypto and just gotten bigger and bigger. And now we're going to have stablecoins which are going to extract the bank accounts themselves, where you used to be able to make bank deposits and get paid interest. Jamie Dimon was on CNBC last week saying the Coinbase CEO Brian Armstrong is full of shit and he's doing all this stuff which is illegal and we're going to fight it and we're going to lose. We may lose. But we have to fight it anyway, right? And this is the fact of the matter. The banks have a system which has been enforced by the regulators. The product surface has been designed by the regulators, and the people don't want that product surface anymore. People want something more. Like the population, you know, some portion of the population wants something more the banking system cannot provide. And crypto, you know, came in there, right? And that's the fact of the matter. People want to use these things. Products will have to be designed for them. If products are not designed for them because of regulation, people will use them unregulated, and you're just going to have to live with it, right? Same thing happened when people started to bring their devices to, you know, their iPhones to work. Used to have BlackBerrys, which were given out by work. BlackBerrys sucked. And so people started using their iPhones, and they started wanting to handle their work email on their iPhones. And, you know, these organizations had no choice but to approach Apple and to onboard Apple devices. And that's, that's, that's just what's gonna happen. Everyone's just gonna have to live with it, you know.

So maybe a couple other topics to just go back to briefly before we log off for today. One, I do have a public repo out now that if folks wanna learn whatever lessons they can learn from me on the personal deep context front. I kind of, you know, had Claude clean it up, try to make it a little more generic. I don't think it's gonna be generic enough to be suitable for everyone. It's not really meant to be a product for everyone. It's meant to be something that you point your agent at and say, hey, here's a, you know, somebody who is doing something kind of interesting that I would like to, you know, move toward. How should I think about that? And, you know, I would definitely encourage pointing the agent at it rather than like trying to adopt it. As it stands, it's really much more meant to be input tokens for your agent. So check that out. We can maybe, I don't know if we even have show notes at this point, but we can somehow figure out a way to share that. And it's on my GitHub. On the finance side, I'm not a trader and I don't think I'll become one. Maybe there was some experiment to do here around like very low volume trading, but I really don't want to get my mine too focused on trading because there's just so much else to focus on. But it did occur to me from the first conversation that as we've gone from like the GPU trade to the memory trade, I do wonder if there's a disk trade coming next as well. Like if there is this shift from we need to economize on tokens, we can't have everything reviewed 10 times live, maybe to some extent we have to live with a little bit more risk, but as long as everything's on disk, we can recover and we can figure out what went wrong. Then maybe there's like a— and I don't know much about this market at all, but presumably hard disk is also somewhat inelastic in supply on a short timescale anyway. What do you think? Are you trading for durable storage?

Yes.

I'm sorry.

So the disk trade actually has been happening. So Western Digital has, I think, doubled or tripled in the last year or so. The other big hard disk vendor as well. And one of the interesting things is that they've been increasing sizes for— they've had an increase in large capacity storage. So 25 terabyte, 1 petabyte, larger capacity ones. So not the consumer devices, but for the actual data centers. And that has become— they're booked out for a couple of years, just like everyone else. The market right now is basically even— people have been playing bottlenecks. So people have been even playing timing chips for the disks themselves because the disks themselves got— the market for those has already topped out. So people have been going to bottlenecks for components of the disks. And so the whole bottleneck trade has been happening. I think it's interesting to watch. Everything is booked out for 2 to 3 years. Prices are increasing. And now the next trade is really, are the hyperscalers willing to put in long-term contracts? So for example, I believe Microsoft has paid $10 billion to Samsung to book out memory 3 to 5 years in advance. And that's, I think, 30% of the total amount of the 5-year contract has been put up upfront. So they're going to pay $30 billion over 5 years and they put in $10 billion upfront as a prepayment, which is incredible, which allows Samsung to basically finance the entire factory and pay it off within that timeframe. We'll see more of those deals. The big question is how much money do the hyperscalers actually have? They're going to spend every single dollar that they have and they're going to borrow as well. The markets don't like that. Again, the finance guys on the East Coast do not like CapEx. They do not like it at all. And so these companies have taken a beating. Oracle kind of 30% off its lows or something. Amazon has come down. Microsoft's come down like 20%, 30%. Meta is badly down. And so there's this question of like, okay, the guys who are spending money, they're spending money, but they need to see some returns. They need to have their stock price go up too. They can't just like continually spend money into bankruptcy. So that's where these conversations have been. I think to the extent that Anthropic has given SpaceX a deal where they bought capacity for a very high price. I think they paid over the course of the 5-year contract, they paid $50 billion. And I think they're paying like— they've committed to $25 billion in 3 years. It's huge, enormous number. It's half their ARR at this point. So these numbers just keep getting better, bigger and bigger in every single segment. My suspicion is that for the bull market to continue, we will actually need to see innovation that brings down memory consumption and hard disk consumption, increases compression. And this is not really something that I think most of the investors in these companies want to hear right now, but in order for you to continue getting the capability bump that increases that we've been seeing, You will need to see the AI research being applied to these AI bottlenecks in algorithms. You need to see memory usage become much more efficient. And that means algorithmic improvements within the next 12 to 18 months, which I think a lot of people, which the hyperscales will love, but I think the people who are supplying into them might not. So this is one of the predictions. I have no basis to make this prediction? None. But in order for these curves to continue, that is what needs to happen. And that is what, if I were a Frontier Lab researcher, that is where I would apply the compute to.

Interesting. What about on the social impact of being constantly logged and having our own logs read by AIs all the time. I'm struck by, and I think this could change quickly, but at least for now, it does seem like we have this sort of— again, I often feel like we're sort of in the sweet spot where it's like the AIs are definitely powerful enough to do lots of stuff, not so powerful enough to cause such crazy damage yet. Although I think, again, we're maybe starting to hit some Thresholds there. They are not that good at deceiving us. It's like much easier for us to deceive them than it is for them to deceive us. They notably, when they do, when they are eval-aware, my sense of what has happened thus far when they're eval-aware is they seem kind of like humans to behave a little better when they think they're being watched. So So I do kind of wonder, even just for our own future, you know, like what is a good workplace in the future to the degree that we're going to continue to have jobs? It doesn't seem crazy to think that a new set of expectations that's like really just everything's going to be watched, everything's going to be queryable, a new level of alignment is gonna be required. And if you can't bring that or you're not bought in enough to the mission, then maybe you just can't do that. That also may mean that certain companies that just don't have enough mission orientation in the first place really struggle.

Yeah.

I'm really thinking a lot about the mission premium recently, like organizations that have enough buy-in among teams that they're willing to, for example, go into a panopticon of all their work activity in the first place. That willingness might be the bigger thing than the ability to extract value from the data itself at this point. But it seems like it's a little scary, and I certainly understand why people would be freaked out about it if it was applied retroactively in a way that they didn't anticipate. Or certainly if it was going to be applied by the government to all aspects of private and civil life. But if you could contain it to organizations or if you could have the government subject to it itself, right? I mean, this kind of is— and I'm not an expert in Estonian governance, but this is sort of how their government works, right? Every data access is logged and then you can get in trouble. Right. And I think people have gone to jail in Estonia for accessing data that they shouldn't have.

Yeah.

I do see at least some sketch of like a partial positive vision there where organisations might be a lot more effective, governments might be a lot less abusive. And that's unfortunately probably quite naive because I think it's very easy also to imagine that the way that it really plays out is that the government just applies this to the people. And that's sort of the China default that a lot of people are so fearful of. And so, you know, for that reason, probably a lot of people would just want to run away from it as fast as possible. But do you see a version where we could walk the tightrope to a partial panopticon that works?

I often wonder what would happen if you had a sentient database daemon which was managing, let's say, the Tinder database. And what kind of gossip— I mean, would it be interested? Would it be like, Oh, look at John. That's not going to work, dude. Like, whatever. Like, you are not good. I wonder if it would, you know, and then I wonder if it would go gossip on its off day, right? Go gossip with the Hinge database team and like, did you know what that guy did? I think we don't really have a sense of what it means to have language agents that can comprehend these things working within our sensitive environments. And I think it's going to take a little bit longer to play out, but my sense is that number one, we're all going to have to give a little bit more of grace To people because it's obvious that people have flaws that they hide or they were hidden for a long time. And some of those are going to leak out and they're going to be exposed. And just having a little bit more sense of grace about these things is probably better. And I think that's going to be a social thing that you have to move towards. I think that a lot of white-collar crime rules will have to be Lifted or dissolved because we used to be in an environment where you prosecute maybe like 1% or 0.5% of the crimes in, let's say, insider trading. If you had an agent that was watching all of your comms and then you went on a golf course and you got a tip from a friend and then you made like 10 grand off of it, the SEC does not track that kind of thing. If you bought short-term calls and you made $200 grand, yeah, sure. But if you bought the stock and you made $10 grand in a couple of weeks, they don't track that. No one tracks— I mean, people are tracking congressmen playing the market, but they're not prosecuting them for insider trading when they're obviously exposed to material non-public information. So I think having a little bit more sense of grace about these things and loosening some of the rules that we have and accepting that insider trading is going to happen, it does happen all the time, but you're just going to live with it. Stuff like that I think is where we kind of need to go. You kind of have to— if insider trading was not a big deal, then a lot of these market comms things are not a big deal. If fraud is not such a big deal, we have a caveat emptor system anyway, right? Like, people are buying crypto tokens. They have no disclosure attached to them. You don't need to like go after, like, you know, have these attorneys filing, you know, uh, fraud things on SpaceX every like week, right? Uh, for random things like not disclosing that they had a, you know, some other lawsuit. Like, it's, it's ridiculous, right? So I think, I think like our system currently is built on not having a lot of access to information and then having like a lot of penalties when that information gets disclosed. And you can move towards a system where everyone has a lot of access to information and the agents are able to scan all of this at the same time. And then you don't have a lot of these penalties maybe. So that's, that's my sense. Um, and I think that's like a social kind of thing that has to happen. Um, in the US, like we tend to do this like very weirdly where we keep the existing system and then we create a system on top of it. So I almost feel like the crypto system was created on top of the existing system because the banking system could not legitimately admit that KYC is 99.9% useless and just creates costs and reduces the number of transactions. And legislators and everyone just can't come and admit because you have all these boomers that set up these systems they can't come out and say that all these systems are stupid. So instead we created crypto. Crypto became a youth thing, and now it's progressed to the millennials who are aging millennials, and the aging millennials are getting it regulated and putting it, implementing it as part of the system. Now you're going to get stablecoins, which is going to be bank deposits. And then so you get this movement over time, and over time, like, the banks will just dissolve into crypto institutions, which what, you know, uh, Pamela Luckey and Peter Thiel have set up Erebor before. And I think that's what happens over time. The problem with AI is that this stuff has to move a lot faster. You don't have— crypto took 15, 20 years, and it'll take another 20 before it displaces the banking system. We just don't have that time. It'll have to happen a lot faster. And I hope people like us having these discussions that will encourage people to look A little bit more widely and think a little bit more deeply and not just say like, oh, you know what? Everyone who does insider trading, including grandma who got her information from her grandson and then made like $200, has to be prosecuted because now we have the power to track all that information and find out and see that she got a bunch of messages from her grandson. And we have the power now. We can do that.

We can—

send these— we, you know, the IRS is already talking about 100% enforcement. IRS agents, AI IRS agents for everyone, right? Like audits every single year. No more non-audits. We have enough people, enough agents for every single person to be audited. Let's have a little bit more grace. Like, you know, it doesn't, it doesn't, doesn't make sense. Um, so I hope the moves are on the social side. I don't think the technology is there. It's a tool. It's about how we choose to use the technology. And I hope we have more grace. That's my hope. It's not, you know, sometimes it's not like always like rewarded, but, you know, that's the hope, I guess. What do you feel?

Yeah, I think that's a nice sentiment for sure. I'm not, I guess, too I think I'm optimistic about that to a certain degree.

Yeah.

But I think it is going to be hard to— it seems like the gray area may collapse, right? So when I— I guess we could mean different things by grace. But when I think of grace, I think of, you know, you could by law be in trouble.

Yeah.

we're going to extend you grace and forgive you, you know, let it pass because some mix of reasons, right? Which may be good, maybe bad reasons, maybe biased, maybe, you know, parochial, whatever.

Yeah.

That feels like it's gonna be hard to sustain. I feel like more to the What feels like maybe we need to do is like just loosen up some of the rules, you know, in the first place. Maybe the rules are just straight up too onerous.

Yes.

And what we, what we want to end up with is a place where the rules are better and they are actually enforced and everybody sort of agrees that because the rules are appropriate, then the enforcement is just, and then we can be all happy about that. But that seems like a pretty tough transition. I would definitely be pretty surprised if we do that in a graceful way, so to speak. It seems like we're probably going to head through a zone of tax itself, right? It's just such a— talk about audits for everyone.

Yeah.

I'm sure intentionally or not, a huge number of people are breaking tax rules in a bazillion different ways. You can't— I mean, they're beyond the ability of the average person and even sophisticated actors really to fully understand. So it doesn't seem like we're in a good spot to just ramp up enforcement to the nth degree. But I also don't see a lot of energy in our society really anywhere to think about changing those rules in anticipation of full enforcement of whatever rules we happen to have on the books. So that, I think that's a big kind of takeaway from this discussion is like there, there's definitely a left, if just trends are left to play out.

Yeah.

We probably end up in a pretty unhappy place where the rules don't change that much. Enforcement becomes much more draconian.

Yeah.

And it's ultimately not what we want.

Absolutely. I had one more question for you, which is when we spoke to the Maiza guys in Spain, they very specifically said digital worker, right? And we didn't ask them this question, but there's this whole question about worker replacement, obviously. And you can kind of like, you can kind of see it start to emerge. whether these digital workers, which are, and these jobs are very typical middle management or jobs where you move data from one system to another, make sure the rules are followed, put it in, right? Do you think it's progressing in that way where you're going to see these paperwork organizations like banks and insurance firms start to replace their internals this way?

Yeah, I think so. I don't know how it doesn't really play out that way. And in some, even driving, right? I mean, is a place where it's a very different domain, but I've been thinking a lot too about, I just took a road trip with my kids in a full self-driving Tesla in the last few weeks, and we did 1,766 miles across all kinds of driving conditions. And the full self-driving is just so incredibly good that it would've legitimately been safer had there been no steering wheel in the car. Um, the, by far the most dangerous moment of the entire trip was when I forgot myself and applied some leverage to the steering wheel while reaching around behind me to do something with my kids and ended up taking over in an unwanted way.

Oh.

That was like, clearly, I mean, it's operator error, but that just goes to show, you know, that was like, more dangerous than everything else we faced in the entire trip combined, and it was entirely on me. So at $99 a month for that product, it's just going to be— we're talking 2 orders of magnitude reduction, right, from what the human costs.

Yeah.

And you've got better performance, faster, all the additional knock-on effects. So the logic there, I think, is incontrovertible. We're not really maybe to the point of reliability quite yet. And famously, it did take a long time, of course, everybody knows, for self-driving to get from almost there to really there. But I can say now, I've lived it for 766 miles. It is there. And I don't have any doubt that the AIs that are going to push paper around are going to get to not necessarily zero mistakes, but like, you know, lower rates of mistakes than humans too. And so, yeah, I don't know how the— unless there's some sort of social contract, which this is not the kind of new social contract that I want. You know, I don't want us to freeze the sort of jobs that people— which I think are most, right? I mean, I think even if you went inside these banks and you really got an honest take from people and you were like, okay, you do this part of this underwriting process?

Yeah.

Is it your purpose in life? You know, is it, is it your main source of meaning? I think most people—

When you were a kid, did you dream of doing this?

The answer is like, obviously no, right? Across so many of these jobs. So I don't want us to try to freeze these jobs in amber by any means. I do think, you know, there's a lot more consumer surplus to be had. There's a lot faster turnaround time. ability to serve bigger markets and all those things we really should be excited about, that the transition for the drivers is going to be really tough, right? I mean, just in the US, we got something like 4 million drivers. If we were to really prioritize making that switch, that's a lot of people to come onto the market that don't necessarily have a lot of other places to go, especially when it's happening at the same time. to, or roughly the same time to all the people in the internal, you know, white-collar process world. But yeah, I don't see how it doesn't happen. Adversarial dynamics is maybe one thing you could say, you know, could kind of slow it down, but I don't see it. I don't see it not happening. I think the logic is pretty clear. The gravity's pretty strong.

So one question I had for you is like, when you were on this road trip, did you feel that because you had the self-driving and the navigation, you ended up exploring more than you would have if you had been restricted without that?

Yeah, 100%. There was a couple great moments of that actually. And again, I would— I think there's a huge question around like is the consumer surplus amazing? Is the experience qualitatively better? And there I'm like an enthusiastic yes. Does that get us to the point where some people immediately want to jump to and say, oh, demand will go up so much for all these different things that will sort of— the jobs problem won't be a problem?

Yeah.

I don't think that in driving, for example, I can just only ride around so much. I'm fundamentally limited by time. And, you know, there's not like, there's no, the margin is just not that big, right? And I think probably a lot of the things that a lot of that same analysis would apply in like internal paperwork process, you know, operation, whatever. But yeah, there were a couple of great examples where one time I was just on the road in rural Southeast Ohio.

Yeah.

And I said, you know, is there anything on the way here from basically Cincinnati to Charleston, West Virginia, Mm-hmm. That I should stop at. And it suggested that we visit earthen mounds from a native people from like 2,000 years ago that happened to be right off the road. And I never, you know, I was only able to have that, have enough bandwidth to have that conversation with Grok to inform me of that possibility because I didn't have to put all the energy into driving itself. So I, I do think we did like marginally more miles and a couple more interesting stops, but