AI:AM — AI Security and Real-Time Content Safety · June 3, 2026

So good morning. It is, uh, June 3rd, uh, 2026, 9 AM, uh, 3 days into the singularity. Maybe, or maybe, or maybe a year, or maybe, you know, a couple of hundred years. Um, Nathan, good morning. Um, and—

Good morning, Prakash.

It's, it's, it's great to see you again. And we have overnight news, um, overnight news, uh, which is President Trump signs the executive order, and what do we know about this?

You know, I'm kind of scratching my head about the whole thing.

It—

as so often, I feel like, with he who must always be named, I feel like there's such path dependency to this, and it's unclear how much staying power it will have, and it's unclear what is really meant by it. So I don't know, I think it's all fairly confusing. But I mean, I guess the, the big headline is, right, they're gonna ask politely for model companies to give them 30 days to review models and render some opinions. It sounds like a very sort of gentleman's agreement right now from What I can tell, or maybe even just a proposal for a gentleman's agreement, because I don't think this would actually prevent anybody from just, A, not participating at all, or B, launching whatever they want to launch despite the considered opinion of the executive branch. Now, obviously, there could be consequences for that. So, you know, who has what leverage, I think, is, is pretty interesting. But we've seen broadly that the model companies do have a lot of leverage. Anthropic has basically weathered the last storm that the government tried to, you know, all the big pressure campaign of everything that happened with the supply chain designation, all that sort of stuff. I think we're just memory holing that now. It seems like, I mean, I don't remember that ever being taken back, but, you know, I don't know. Again, it seems like we're just kind of going to pretend that never happened. So what's— I think, you know, it's, again, it's— we have to see how this stuff shakes out. The talk is that the benchmarks that they're going to develop themselves are going to be potentially classified, which is a little bit weird. Although I'm not sure if I'm like so outraged I mean, I do think we have a problem in general with benchmarks being overly transparent in some ways, and that's why organisations like SCaLE and others have created these semi— and even the ArcAGI Prize. And there's all these different things that have kind of the— you get a few sample problems and then there's a lot held back because we just can't trust the results if we put it all out there. Now, they could do that without making the whole thing classified, but don't know, that's almost kind of, I guess, reflexive for the government to just start with a posture of classifying things. So there's been some interesting commentary online about, including from no less than a friend of the show, Dean Ball, who said we may be headed for a situation in which the government will be testing models you can't use on a against a standard that you're not allowed to know about. And that does seem strange. And I agree with him that by instinct, libertarians shouldn't like this. And I definitely consider myself pretty much a lifelong techno optimist libertarian. But is it that different at this point from Scale AI having a semi-private benchmark? And is it, given that it's just sort of this polite asking, is it really something that is going to throw the industry into a, a crazy vortex of secrecy or, you know, self-dealing or whatever? I don't know. I'm not that worried about it yet. I'm, I'm a little— I just kind of— I guess I'm just kind of of a mind to wait and see because surely something like this needed to happen at some point. You, you would expect that a government that has any sort of Has its act together to any degree would want to do some of this testing. They would want to be in the room as decisions are made and know how decisions are being made and have some hard facts to make sure that they feel good about what the companies are doing.

Mm-hmm. Mm-hmm.

So I don't know, it seems not entirely unreasonable, certainly. And I guess we'll see if it morphs into something that causes a lot of problems. But right now, I I slept just fine last night having consumed what discourse I did. What do you think?

Um, well, uh, regardless of the EO itself, um, just the like staging of it, right? The, the, the timeline. If you look at the timeline, I think they wanted to sign the, the EO was already— I think we had like, I think the Mythos release, the Mythos release kind of the The Mythos hype really kicked off this whole thing. You had at the one hand, I think even before that, the whole Anthropic versus Department of War thing. Then you had Anthropic about to be excommunicated. And then a couple of weeks later they have the Mythos pre-release, then the preview goes out and then that freaks everyone in government out. And then that freaks everyone in government out enough that an EO is formed, right? The first EO draft is formed. President Trump almost signs it and David Sachs pulls it back, right? David Sachs pulls it back and he says, this is going to slow everything down and this is going to slow the economy down. So President Trump pulls it back. And then you have this period of, I think, negotiation between the two sides. And then you have the preemption which is the states start issuing regulation. So Illinois last week on Wednesday issues regulation. And one of the things that is striking is that J.B. Pritzker in Illinois is a presidential candidate 2 or 3 years in the future. He's a billionaire. He can fund his own campaigns. He doesn't have the stain of antisemitism that some of the other Democratic candidates have because he is of Jewish descent. And so he's a much stronger candidate than I think some of the other candidates that the Democrats might put across. President Trump has had his eye on JB for a while, has called him a bunch of names. And so JB Pritzker puts forth this bill, and he's one of the only few people in power in the Dem side that has actually done something at this point because Bernie Sanders is not in power. He has to— legislation has to go through Congress. AOC, same. Newsom pulled back the AI bill in California. So JB is now the leading anti-AI candidate because he's actually done stuff. And so he's done— I think he's making a bet for 2 years in the future, which I think is a very wise thing for him to do. And on the other hand, Congress has failed to have a bill that preempts state regulation. And it failed primarily because the Republicans themselves were not able to get their act together. Marsha Blackburn in Tennessee, Nashville does not like the music like AI, and they're extremely afraid. The other thing that I always point out is that in the political economy of the US, you have all of these states that have lost out to Silicon Valley. You have like the entire, you know, the music industry in both Nashville and LA, which has lost out to Silicon Valley, uh, through Spotify and these, these, these other tech companies. And so you have this like, oh, you know what, the Silicon Valley guys are taking everything, right? Um, and so this is kind of like, you know, uh, the other states kind of putting, putting an act— putting a united front together. Um, and so I think what's hap— what ended up happening is that since the Republicans were unable to get their act together The states went out, and now, you know, this EO comes out. The EO is a midstream kind of like trying to preempt the states. And now you have Ted Cruz on the other side. Ted Cruz is like, you know, let's do something about this because we cannot— you know, AI is developing rapidly. This administration is right to recognize the cybersecurity risks posed by advanced models. Now it's Congress's turn. We must address catastrophic risk without ceding ground to China or restricting Americans' free expression. So you have this tick-tock of like, you know, the states, the executive, the Democrats, the Republicans, the state economies, all of this stuff kind of intersecting with each other, colliding. And let's see whether or not we get the federal preemption. I am not that hopeful. It's an election year. And, you know, people are— I think legislators on both sides are unwilling to do things negative. They want the positive things to happen. They want to claim credit for them, but they don't want to be blamed for any of the negative things. So let's see. Let's and you know Ted Cruz is not. I don't think he's up for reelection this this year. So the the six-year cycle for the senators gives him some breathing room. So yeah, let's see what happens. I think the EO itself is non objectionable, but the EO kind of creates a licensing regime. I know people think it's voluntary, but it's not voluntary when You know, if you don't do it, your other peers, like, you know, if OpenAI doesn't do it, Anthropic and the other peers will be like, oh, look at OpenAI, they're so bad, they're terrible people, like, et cetera, et cetera, right? So the media and the social media will take care of the enforcement, even if the EO itself does not have enforcement. Open source guys, I think one of the main things that people were concerned about was that open source would get regulated and the smaller companies would have to do stuff. I don't think this affects the smaller companies because to some extent, the smaller companies have a Streisand effect where if they are seen in somewhat of a bad light, if you're a $10 million company and the president talks about you, even if it's in a bad light, That's pretty good for you, right? People are concerned. What you're doing is important enough that important people are concerned. So maybe it works out okay. Maybe it's not a bad thing.

Yeah, it feels like— I mean, we talked a fair amount yesterday about how I came away from my weekend at the Recursive event with the negative update that the plan for keeping recursive self-improvement on the rails largely boils down to chain of thought monitoring and maybe a little bit more advanced monitoring strategies than that, but not a ton. But then also with the positive update that it seemed like the Overton window among people at frontier companies has expanded at least a bit. to the point where they're willing to talk about, again, the possible need for a coordinated slowdown. And it seems like there's a bunch of things that are kind of trending at the same time toward, dare I say, like a kind of healthy political climate with OpenAI. I mean, their situation is obviously kind of confused, right? But they They ended up supporting this Chicago bill, which I'd say my read is that it's maybe even a little more demanding on the companies than SB 1047 originally was. Well, there are a lot of versions of 1047, but the final one that got vetoed certainly didn't require this third-party auditor access that is the kind of core of the Illinois concept. And they ended up, you know, they came around to support it. Um, they seem to have sort of distanced themselves from a lot of the PAC activity funded by Craig Brockman, which, you know, is now being, uh, disclaimed as just personal capacity, just personal capacity. And we're not like in that different of a spot. You know, you look at what the EO does and you look at what this Illinois bill does, and then the Connecticut one, which is— I would have to study up a little bit more on the mechanisms to really hone in on the differences. It seems like— I know that the organisation Fathom, who I had done a Cognitive Revolution episode with a while back, that they're really pushing this idea of a sort of private governance market. where the state or the federal government, whoever would kind of manage this regime, would be in the business of defining safety outcomes that they want and then empowering private auditors, quasi-regulators, to be the ones to actually interface with the companies and determine that they have indeed upheld those standards. So it seems like maybe the difference between the Illinois thing and the Connecticut thing is this slightly different role for the state, where in Illinois it seems like the state is just saying you have to have third parties do some review. And in Connecticut, they're maybe saying we're going to give you kind of a pre-approved list of organizations that can do this review.

Mm-hmm.

And that might not be quite right, but that seems like it's probably roughly the difference. Not that big of a difference. And that's also not that big of a difference as it compares to the EO. There is, of course, you know, something quite different about having a federal agency do it versus having, you know, a meter or an Apollo EVows or whatever do it. But it's still basically the same structure. You know, it's basically just taking a beat to say, we want to go in, get under the hood with this, see what's going on, make sure we are comfortable with it. And it's not a super long time, and it doesn't seem like it's going to be extremely onerous. So, and then the gnashing of teeth around it has been fairly minimal, right? Like, we haven't really seen anybody freaking out from any of the companies. So I'm not sure how we ended up in this spot where it seems like everybody's a little bit more chill and on the same page when the requirements have— they certainly have moved up a bit relative to the final 1047. But it sort of has happened. You know, maybe I'm just not in DC. I'm sure there were some, you know, intense moments of wrangling and what have you. And certainly there's got to be some drama behind the scenes at the White House. But I think that's just kind of how, again, he who must always be named, you know, court politics kind of works there.

Mm-hmm.

So yeah, I don't know. Overall, it feels like surprisingly healthy dynamics compared to what I would've expected as we hit the end of the 1047 cycle, you know, whatever, almost 2 years ago now.

I, I, I, you know, let me know what you think, but I feel from both Anthropic and OpenAI, they have calmed down a little bit because I think it's very clear now that number one, they're in recursive self-improvement. Both, both firms are. Number 2, that China has fallen behind and the chips kind of ban on China has worked and they have fallen behind significantly. And number 3, that alignment has worked more or in the target area that they wanted it to work. And so they've gotten a little bit more confident that they can pull this off. And I think those 3 things, and they've also, I think, settled down to the fact that both of them are going to exist, that there isn't going to be just one firm out there, which is going to be the dominant AGI. And I think that there's a little bit more comfort that it's going to be at least 2 and maybe more. And I think those things together have made them, I think, more sanguine about regulation in general. And if you look at what the expectation is for any kind of regulation, number one, I think the audit stuff, there's no way that I think someone externally is as Knows what someone internally does or knows at the same speed. I regard the auditing more as assurances to the public and the government. That's number one. Number two, it's a little bit of bribery because you can move all of your, you know, white-collar labor that's being laid off into assurance and audit, which is— which can be an infinite, infinite sink of like, you know, let's evaluate AI models. Like, All humans do all day is evaluate AI models. And the more AI there is, the more models there are to evaluate. So that's a great sink of human capital and labor, which is required to make the models work well. So I think those 2 things are there. And it's also kind of like, again, most of the time for audit, who ends up making money is a bunch of lawyers. And those lawyers are all affiliated with either the Dem or Republican Party or the state. And so you manage to shuffle in funding into these guys to keep them happy and to keep them occupied and to not have them sue you all the time. So I think that, I mean, it's a little bit of a cynical view, but I believe there's also good in there that you get people to look at the stuff, but there's also the cynical part that you are paying off the guys who are most likely to be energized to challenge you. So I think there's that aspect there. So again, this combination of them getting maybe a little bit more confident that they're not going to die immediately and that China's not going to just supersede them, I think that's been helpful. But what I expect to happen is, and what we always expect to happen in regulation, is that the regulation slows down release. And so the question for me is that how much do the releases get slowed down? And is there any other impact besides just the releases getting slowed down? Does that mean, for example, that China does catch up or the economy takes a downturn because the models are not good enough to— that the models that have been released are not good enough to do the tasks that people expect them to do? So there's this little bit of question there that in my mind on what happens. you know, as a result of the regulation, as a result of a slowdown in releases.

So of all those points, I— the China one really jumps out to me, and I had a note on that from the Recursive Experience as well. It was really striking there how little China came up, and that was a big difference from say, you know, going to The Curve, um, last year, or certainly the year before. The first instance of The Curve that I went to had a tabletop exercise that was a simulation of US-China, you know, race dynamics. And the actors were like US labs, the president, Congress, the CCP, what have you. Uh, and the scenario was basically like The US is ahead, but the Chinese just stole the weights for the latest model and they can't run it as much. They don't have as much compute, but they do have sort of qualitatively similar capability, just not as much ability to scale. And that US-China dynamic was really at the centre of the exercise. At Recursive, just more recently, there was another tabletop exercise But this one had— and I didn't participate in this one because there were too many other talks that I wanted to go see, but I went and just kind of sat in to see the setup of it a little bit. And this time it was really just, you're at a lab and now recursive self-improvement is happening and there's different actors within the company and we're going to play out, you know, all of our roles will be sort of this internal dynamic with really no major role for China. There may have been a little cameo. for China. But the vibe was, I think you're absolutely right to put your finger on the confidence that they're ahead and they're probably going to stay ahead and that there is some buffer that they can absorb a 30-day pre. And of course they do. Another interesting question for this would be like, They do their own reviews, right? And how long do those take? I think it obviously varies across companies and probably varies across models, even within companies, depending on things like how much of a difference there is between this model and the last one. I was just listening to 80,000 Hours podcast last night with Rohin Shah, who leads a lot of the safety efforts at Google DeepMind. And he was saying, pretty reasonable in a lot of cases to think capabilities advances are pretty smooth. You do see sometimes thresholds crossed, but generally speaking, if you have a well-designed thing and you've got tests that you're confident will come back positive before the actual risky behavior or capability is achieved, and you have some buffer there that you can be that you can count on, then it's pretty reasonable in his mind to say, hey, it's only been a month, we've only done a little bit more, we haven't really scaled tremendously, and the tests didn't— the alarm bells didn't go off last time, they're not going off this time. We don't have to do a crazy whole model card from scratch for every single point release. So anyway, it varies is the point in terms of how much delay there is between training completion and launch anyway. But I suspect that a lot of this 30 days can be overlapping with their own internal processes because they can— it's one thing to say, is this model capable of X? Then it's a somewhat different thing to say, as we deploy it with all these system-level filters and safeguards and whatever else, Will that actually happen in a user-facing way? And I think the government, at least at first, is probably going to be more concerned with just absolute capability. And that can probably run in parallel as long as they're not hitting those super key levels. That can probably run in parallel with their own kind of system upgrades and whatever else. So it doesn't feel like it's going to slow things down too much. Um, but again, you know, we're only one stroke of the pen from, uh, a change to the order. And I, I honestly, you know, I'm interested in your odds-making, uh, sense on this, but like, will there be an update to the EO in the rest of this year or in the next 12 months? I wouldn't be surprised at all if there's a revision, you know, because this very much to me feels like a V0, you know, kind of rough draft of what the ultimate regime is going to be.

So I think the executive orders are not that interesting for me as much as Congress is. And I think that's the main game. The main game is what do they think that they can get passed and how can they pass it? Because they need a little bit of buy-in from the Dem side. They don't need a lot, but there's at least like 5% of the Republicans who don't like AI as well and who want more states' rights in determining what AI does. And those guys will collaborate with the Dems who I think are largely going to be anti-AI. I think it's just too dangerous. As a Dem, I think, to appear pro-AI once I think the progressives, which are like 30% of the party, are anti. And so I think, you know, it might be that the Republicans are like, you know what, we're going to lose, we're going to lose the House. And let's just pour everything in and get it passed and then, you know, deal with the consequences later. So yeah, I think Congress is the main game. The EOs, Trump can revise them as he wants. And he's a very active investor. So he can do stuff on the fly. I mean, he can change policy in a tweet. So I don't think it's a big deal. He's very adaptable. But I think Congress is the main game. And a lot of what he does is about massaging the media and massaging Congress to get what he wants. So yeah, I think that's the main deal. I suspect what's going to happen is we're going to see a real slowdown because people, now that the EO is out, the EO provides a venue for other people in government to say no, right? Because then the models come out. And so you can just imagine, like, Meetha's preview was given out to the agencies before they announced it, 60 days before they announced it, the agencies used it and they found like holes all over, security holes all over the Department of Defense, all over the NSA. And they just don't have enough contractors to patch everything. And the holes keep appearing and appearing and appearing every like, you know, day more holes are found and they panic and they say like, look, you guys cannot release this. You're not allowed to. And, you know, within 60 days, there's, you know, open source models which can match that. And, you know, as a result, the rest of corporations like Microsoft, etc., etc., do not have access to Mythos and are instead attacked by, you know, open source hackers, attackers using open source from North Korea. It's a very, very, like, easy thing to see, because even now, there are not enough security people patching the Mythos, you know, bugs, and they've had to delay release of what they actually found, right? Even it's been 30, 45 days now. So I think that's the natural thing that will happen, which is the government will not be fast enough on their side, and they'll ask for delays. And I think that is going to have impact on security for the rest of corporate America, which holds a lot of important data, and which also collaborates with the government for a lot of governmental functions. So You know, there's going to be that downstream. And I think that's what's going to happen. You're going to have this cascade, which is, I think, what people wanted to avoid in the first place. But now that it's out there, it's going to happen. And then after the first cascade, they wise it up a little bit. All right, we can't actually do this. Like, you know, government needs to patch quickly. Corporations need to patch quickly. We have to get this stuff released out. So we're going to have to go through that. you know, uh, that learning process now in government, uh, and then once they have that learning process, like 6 months later, maybe, maybe things shape up. Like, I guess that's my, you know, uh, pros and cons and, you know, the way government works. And, you know, so it is, it is, as, as, uh, as one of my mentors used to say, it is what it is. It was one of the most annoying— it's a, it's a real dadism. In any, in any event, let me—

Yeah, perfect segue in a sense. Yeah, perfect segue in a sense. I mean, one of the big questions I have there, which I think our first guest today can hopefully help us understand better, is if it's so easy to find these bugs, why is it not easy to close them? And, you know, what exactly, if you're correct, and they might have a take on this, if you're correct that we're sort of already seeing delays because it's just taking too long to fix all these issues. Like, what's going on there? So with that, let's get into it.

Hello, hello. Nice to meet you.

So our guests today are Tal Hoffman and Yanir Zarimi from Enclave AI, which is building what you might call is a cursor for security, an AI agent that does not just scan code for hundreds of theoretical issues, but tries to find the handful of vulnerabilities that someone could actually exploit. Tal is an ex-Unit 8200 engineer and the CEO of Enclave AI. Unit 8200 is the Israeli military's cybersecurity arm, I guess, and a lot of very prominent cybersecurity companies have been formed from ex-Unit 8200 engineers. Including Wiz, which was bought by Google. Janur brings the attacker's eye. He has found serious cloud and Microsoft vulnerabilities, including cross-tenant bugs and agent-related failures where the security check existed but sat in the wrong place. The reason this conversation matters is simple: the software is increasingly being written, reviewed, and shipped with AI in the loop—what we call the slopocalypse. If AI writes the code and another AI checks the code, then the idea of independent security review starts to become the whole game. They recently reproduced the frame, the same FreeBSD zero-day Anthropic's Frontier model found by using Sonnet 4.6. So you could look at it as replicating Mythos with a model which is 100 times smaller. So the question I want to get into is not just which model is the smartest, but where is the real edge now? Is it the model itself, the harness, the tools, the workflow, or really is it the humans using the tools? Because, you know, you have elite people like Tal and Yanir who are more able to use these tools to detect the vulnerabilities. So Tal and Yanir, welcome to the show. Thank you so much.

Hey, Tugbir. Yeah, great intro. I think our take on this matter is that it's kind of a combination of everything. I think it's evident that the harness is becoming super important. The harness, the context that you actually feed into the LLMs. This is why we've been able to produce the same output. It also produced with less good model or a worse model, so to speak. So I think it's a combination of everything, the harness or the infrastructure and the context is, I think, key. And it's a matter of knowing what to look for, where the interesting bugs usually are. And this is Yanir's specialty. This is what he's been doing for a while for a living now. So we kind of bake his knowledge into Yeah.

Can we dig in specifically on this Mythos versus Sonnet doing the same thing bit? The discourse around this has been, as I'm sure you guys have experienced, that Anthropic comes forward and says, hey, we have this model, it can do all these things, it's a big deal. Everybody freaks out. Then folks like you and some others step up and say, hey, well, we were able to do it too in a slightly modified setting. And that setting is like, well, we've got good harness. And I think the phrase was like some guidance. And for somebody like me who is just learning to manage my passwords effectively in 2026, it's not really clear how much work that some guidance is doing. I've seen takes that range the full spectrum from these guys are just talking their book because, look, the whole thing is if you had to give it guidance, you kind of gave it the hint, you gave it all that mattered, right? It's one thing to say, hey, there's a problem in this file. It's another thing to say, go look out at this giant 35-year-old repo and find issues. So I would love to understand that at a deeper level. What kind of guidance did you give? How much did it matter? You know, in a world where you didn't have the hint that this thing had been found by Mythos, like, would you have been able to do it? What would it have taken? Yeah, deconfuse me, please.

No, I think it's— yeah, you want to take it?

Yeah, I think I really like this question. I think why I want to pick this up is that people ask Well, we have this new model now. What what does it mean for cybersecurity? What's the the day after? What does it look like? So in terms of cybersecurity models and the amount of knowledge you need to steer a model, I mean today we measure cybersecurity models on exploiting. So it's straight up they give you a vulnerability description. and you give it the code and it tries to exploit it just off the prompt, one prompt and one shot. So, but when you combine human knowledge with it, it becomes like something much more deeper. People are not really measuring like what does it mean to find a vulnerability from zero. So most like evals today that we see Anthropy coming out and saying, well, We have this model that is really great with cybersecurity, and it is great with cybersecurity, but in a very specific area of reproducing exploits from known vulnerabilities. The description sometimes is as clear as this function in this file has this buffer overflow, even sometimes with the line numbers. So it's not about discovering vulnerabilities from zero, it's about exploiting them. Um, my take personally is that there are a lot more vulnerabilities that are interesting even before we discuss, um, exploitation, um, because models can reason about, um, code. And my take is that if we can convert our knowledge work as cybersecurity researchers— what do we actually check in a system— into natural language, we can actually make models actually reason about this general security model of a system, less of trying to reproduce specific memory vulnerabilities.

I agree. I think, to your point, I think, Nathan, it's a valid argument. I think the guidance part is, to some extent, the process of productizing LLM-powered systems. security research. Even Carolina Informatropics' way of finding stuff is they look at specific files with specific guidance on what to look for. So it actually is a matter of, okay, how much would it cost to find all those FreeBSD vulnerabilities at the end of the day? But definitely guidance here is, I see why it feels like cheating a bit, but it also is the way, if you want to productize something and you want it to be useful, it's something that's actually been baked into the product, making it just more efficient at finding important security bugs versus, hey, Claude, find me security issues, which will then just yield a bunch of inconsistent results. Most of them will probably not be as interesting. So it's about like knowing what to look for, but also in like comparing them to known bugs, like we've like done with the guidance that we've provided, but also a matter of like thinking how Yanir's brain would go about, hey, how am I finding interesting stuff from scratch that's not memory corruption, that's more of a logical bug? For instance, we just yesterday We've published a research piece that Yanir and the security team, security research team found. Essentially, the bug is developers published the Microsoft Android apps with debug equals true or is debug true to production. Very simple bug, but those are the kind of the— The bugs that eventually sometimes break the news. And the bug let us get access to your security tokens. We could read your emails, we could read your files, et cetera, for a very simple, from a very simple security logical bug that's not as novel as a memory corruption bug, right?

So, I have a question for you. So since Carlini, Carlini was up, you know, I think just before the Mythos release, and he said for the next few years, at least the next short period, we have a balance, the balance of power is shifting to, you know, attackers from defenders. And this is a change from the last, you know, a couple of decades. And he said, but, you know, if we get through this period, then, you know, things will be good because we'll have patched most of the things. How has your work changed in the 60 or 90, you know, 60-plus days, or, you know, since the Mythos release and since all of these bugs started getting exposed, you know, in Firefox and FreeBSD? Some of them are 20-year-old bugs, and which, you know, enable very serious exploits. So how has your work changed in the last 60 days? Are you guys running around everywhere, you know, working with clients, trying to fix stuff? Like, you know, is there stuff breaking all over the place? Are you noticing like North Korean hackers all of a sudden having, you know, these new capabilities? Like, what have you noticed in the last 60 days or so?

Yeah, frankly, I think MITRE has done a good job from a sales perspective to most cybersecurity companies because it created a lot of panic in the industry. My take is that maybe short-term something has changed, but longer-term or midterm, nothing fundamentally changed the dynamics between attacking and defending. It's still a mouse and cat game. It's always been that way. And yes, the attack surface is growing. Attackers have more tools. It's scary, but also defenders get the same too, so I kind of agree. Um, but it's still the same game at the end of the day. Um, so that's, that's my personal take. Barriers have gone lower for both sides of the equation, which is good. Um, yeah, this is how I think about, uh, these things. Uh, Yaniv, if you have something to add?

Yeah, I guess, I guess it's still, you know, um, do you have motivated attackers? If you have motivated attackers, things can happen. I think it is still somewhat remains, there's still a moat, like you still need some skill and understanding to hack today. It's not like magic that you can just tell the AI, hey, please hack this bank or something like that. It's, we are still not there yet, but it definitely accelerates the skill of a skilled attacker like 10x, 20x. It's really— A meaningful increase of output.

Yeah, I think to add to that point, I think that the thing that did change is the amount of AI-generated reports in bug bounty programs. This has become a massive problem for the industry. You've seen Linus on the last, I think last week's or 2 weeks ago's release candidates kernel discussion saying that it has become unmanageable to handle those reports. So I do think that the thing that kind of does change a little bit is the focus from a report or a finding to, is this exploitable or not? This is becoming the interesting bottom line. If it's not exploitable, then it's not as interesting. It's a finding, it's not a vulnerability. So you kind of need to differentiate between the two.

So I'm confused about why this would become unmanageable in the sense that maybe it's just a cultural thing. But as it stands today, I've got agents coding all around me all the time. And it sure seems like my ability to fix issues, whether they're logical bugs or whatever kind of issues I want fixed, has Entered, you know, such a qualitatively different regime lately that I'm still adjusting to it. So I hear you in the sense that, okay, yeah, there's an overwhelming number of things coming in, but why can't we just sic our coding agents on all these things? And, you know, why does this distinction between a bug and a true exploitable vulnerability really matter in the presence of coding agents? Like, why not just close them all? You know, is that really so difficult or expensive or slow? Like, what makes that not a viable strategy?

My take is it's definitely possible. Technically, this is what we are here to help with as a company at the end of the day. But that's not to say that it is easy in the sense that, like, proving exploitability is much harder than finding a potential bug because you need to make sense of the architecture of the cloud and runtime. the specific deployment for that specific customer, et cetera, et cetera. So, there's definitely a lift, but I also agree that you could and you should probably use AI to fight AI in that sense. Like, this is how we think about these things. But yeah, exploitability is not easy, not right now still. If Yanir, you want to elaborate?

Yeah, what I see is that, um, how security, the kind of flow goes in a large enterprise is when the security team comes up with something, they send it to the engineering team. The engineering team needs to kind of give their feedback on it, and then they kind of negotiate because the engineering team has their own release dates for their own stuff. And then you kind of give them a new task they need to— sometimes it's like a deep design flaw that you can just patch and fix. So sometimes they have to negotiate, you know, when to patch it, how to patch it, is it important enough to patch now. So it becomes like a matter of prioritization and like kind of a business aspect comes into it. But generally, there needs to be some way to prioritize like what needs to be fixed now and what can wait later.

To what extent do you think it's a feature and not a bug? And the thing that I go back to is that, you know, Nathan and I were having this discussion about enabling our agents, and most of the enablement has to do with authorizing them to use, you know, your Gmail or your Slack or your other items. And it's also, interesting because we, you know, it would be very hard to do in a fully baked corporate environment. Like, you would not be able to set up an OpenCLaw on your own inside a bank. Like, it's just not going to happen, right? So, to what extent are, like, these agents requiring kind of authorizations that are necessary for them, but that is what creates the bug? That's what creates, like, this kind of authorization kind of gaps. And that's what enterprises are trying to solve for. Their executives want to use these things, but the enterprise just isn't ready yet. And that's where the bug is. The bug is not in the software. The bug is in preparation.

Yanni, what's your take?

So, you're asking about security of AI agents in general. Um, I think the world is like not really there yet in terms of agent security. Like we don't have the principles set up yet. Like there's some, uh, but I think as somebody who's designed AI agents extensively and can kind of understand how they work, it's not, uh, an easy task to secure an AI agent and protect like from things like from prompt injections and all of that. But we definitely have like basic principles that, you know, most people can apply to have better security around their agents in kind of limiting what information they can access or, um, what can they do over the network, or giving them like limited tools. So there's a lot of, uh, a lot of things to play with when you want to secure, um, agents, but I don't think we still understand the scale of it right now.

Yeah, I think getting OpenCLaw into your organization might as well be a disaster at this point because, as Yanir says, it's still— we haven't caught up to the adoption curve from a security standpoint.

And when do you think that organizations— what do you think organizations need to do in order to get there? Because, you know, Alex Karp at Palantir has this idea of the ontology, and it's a very, very big word, but all that means, I think, for inside, like, the Defense Department is who has the right to see what data and who has the right to execute which action. And it's a very— like, he uses a very big word, ontology, for just, like, auth. But that process of building, identifying, and building out those chains of command, over the past, like, 2 decades, really, for Palantir is what has enabled, I think, them to deploy, the Defense Department to deploy, you know, these models very quickly because they know exactly what the models are allowed to do and what the models are allowed to see, right? So, how do you think that works inside corporations? Like, what do corporations have to do in order to enable agents?

Yeah, I think, from an authorization and access standpoint, it's exactly that. It's mapping out the ontology, not using that terminology, but understanding the ontology, understanding the structure, and understanding the business impact of if this were to get exploited, what would be the effect downstream. It could be very simple, very naive things like the debug equals true bug. So, I think that's that. And I think it's mostly right now sticking So, all the principles like zero trust, segmentation, you know, the same old principles. I think this, at the very least, you need to be able to say that if this were to get exploited, the blast radius wouldn't be as impactful as it normally would. I think this is the first kind of defense that you need to put in line. And then obviously authorization is based upon ontology, right? Who needs to have access to what? So you're going to see a bunch of vendors, you already see a bunch of vendors trying to kind of solve this problem because it's becoming a real one.

When Mythos comes available, which we understand now might be pretty soon, how do you think it's going to change the way your customers want to use your product. We have stories all the time now about people blowing through their token budget in a few months or starting to think, geez, this stuff is really adding up. Anthropic's top-line revenue obviously reflects that, and that's all before any sort of general availability for a Mythos-class model. But my sense is that for things like this, People might still be very willing to pay. If your value proposition is for the Sonnet plus Harness thing is like, hey, you can spend maybe 2% as much with this setup versus what you would spend if you were doing Mythos. How many customers do you think say, great, we'll take the Sonnet at 2% the cost versus how many say, You know, this is— I just don't want to be reading about myself in the newspaper, so I'm gonna just pay what it takes to have the very best thing. It seems like willingness to pay so far has been really high, and I don't know if we're hitting a limit to that anytime soon, but I'm sure you guys have started to ask customers about this. What are you expecting?

I think the novelty is gonna wear off soon in the enterprise setting. And you mentioned costs. Costs are becoming huge, a huge thing for enterprises. I think Factory AI just yesterday released their router, their model router, and it got echoed really well. And I think Cognition has been doing this very well. I think being multi-model and being able to use the right model for the right task versus throwing Mythos at everything, which obviously is a good model, but you kind of do want to use the the best tool for the job, but also the right tool for the job. I think enterprises and CFOs are going to put a lot of emphasis on that. For as long as you can show that, you know, you get to the same results with lesser models with the right harness, right? If that comes obviously as a trade-off, then maybe they would prefer to use the stronger model. For us, it's about being multimodal, getting them the flexibility to use whatever they want, but also do aspire to be cheaper, to use the right tools for the right jobs.

One question I have there still though is like, how do you know if you're getting the same results, right? I mean, it's one thing if you're in a sort of controlled benchmark-style setting, But it's another thing when you're a financial institution or whatever, and you're like, have I done everything I can such that I can sleep well at night knowing that there was nothing— even if something bad happens, there was nothing else I could have done? It seems to me like we may still be in a spot where people are like, how could I know, right? Unless I, like, sic Mythos on this. I mean, is there a sort of way outside of a controlled environment when you're actually talking about real enterprise systems that people can get that level of comfort that they are getting as much value from a cheaper solution?

I think Yaniv has a strong opinion about benchmarks and evals, so I'm going to let him chat about this for a moment. But I'd say about Mythos, we all need to realize that it's probably a very good model. But it's no magic and it would miss stuff. And it's not like if you were to procure Mythos or GPT-5.5, then you are all covered and you are good to go. That's not the case. So yeah, I kind of want to put, like, again, the novelty would kind of wear off soon, I believe. I believe it's probably a very good model, at least the thing that we see online. And Yanir, if you want to talk about, like, assessing models and benchmarking and Your take?

Yeah, so, um, my take is that we strongly need to depend on, well, harnesses and actual human knowledge is much more important than the models. Um, Cyber Gym, like the most famous cyber eval, the top, uh, score right now is, uh, by the Microsoft, uh, multimodal setup. They used, um, Opus with Sonnet and GPT-5.4, and they got a score that is higher than Mythos. So what we see is that cheaper models can outperform more expensive or smarter models if we just optimize, you know, the knowledge or the, um, harness around it. So I think there's a lot of place for humans with real expert knowledge. Let's remember that, um, how to research software is not like a really documented process. Uh, it lives in the minds of humans who have been doing this for years. And it's like, just like lawyers do, uh, AI agents today, there needs to be somebody sitting at it, looking at the results, and kind of having taste, like what is good and what is not. Like, at the end of the day, there's somebody behind all of those systems that is has to make the judgment call if the quality is up to standard or not.

And they will be accountable if something goes wrong, right? You cannot fire an AI. You need someone to blame at the end of the day.

Yeah. So I think as a result of your work, you've spent a lot of time inside the cloud provider's firewall. And looking at live agent infrastructure as an outsider, one of the questions I've had is, we have these tools that are working inside these firewalls. And let's say you have a database daemon, and your database daemon understands natural language now. So, as it's writing to the database, it starts thinking to itself. Right? Like, for example, you have a database daemon, you know, basically managing Tinder inside Tinder. And it's like, hey, you know, John, that was not a great thing to say to your date, like, et cetera, et cetera, et cetera, right? Like, what do you think? And also, you've had, I think, some cases where, I think there was one, I think, security test, you know, red teaming, where There was an agent sitting inside a biotech company which decided to, you know, which said, like, reading through the emails was like, oh, you know what, these guys are, you know, publishing bad test data on their drugs. We should report this to the FDA. And it started to, you know, form these emails and tried to, you know, email the FDA as a whistleblower.

Right.

So what is— what do you think is this privacy contract that, you know, these agents should have? Right. Because these agents are— we are losing a lot of our privacy here because now you have these agents who are managing the infrastructure and they're actually reading all of this stuff. Right. They're reading your dating history and like what you said, like when you commented and, you know, your emails and like You know, when you ask your CFO, like, hey, do we really need to report this to the public? Or, you know, can we just kind of like manage this for this quarter, right? Like all of these things that we kind of like have these like soft asks, you know, now, you know, someone is reading them. Like, what is this? What do you think? How do you think the privacy social contract should be when you deal with these AI agents?

I think it's a great question. I think ultimately it goes back to controlling agents and then to your point about ontology. I kind of like to think about AI agents in that scenario as to say I have a new employee, human employee, what could go wrong with him if he kind of goes rogue? or messes up? And how do I kind of avoid him messing up, you know, destroying everything? So it's about like limiting access. That's the same old principles essentially. Same about privacy. You know, it depends on the type of agent, but you kind of do not want to share necessarily PII or private information. to their own agents. It still has to be about access control and zero trust. It all goes back in my eyes to the same principles. It's that simple in my eyes. This is how I see it. It's that simple, but it's also very complex to implement, right? But Yaniv, I'm curious, what do you think about this?

Well, it's always, you know, about your approach. Agents like OpenCLAW, are designed to be very powerful, doing everything all at once. You can design your agents in very specific ways to limit their capabilities, what they can do. It's, I think, requires a bit more of a— more effort, but it's, I think if you're employing autonomous agents at an enterprise, you need to actually be aware of the risks that you carry with it. Like, we take care of— we take, we take it very seriously when we are designing AI agents. Um, the impact, what they can do, what, what's the blast radius, is something that's always on the top of our minds.

And you need to be extra careful with wide access. Aid access is one thing, wide access is different.

We're just about At time, and our next guest is here, but I just want to give you one more real quick one. When it comes to the sort of system-level mitigations that companies like Anthropic are going to place around a core Mythos model, how easy or difficult do you think it is to distinguish between defensive cyber tasks and offensive cyber tasks, especially if somebody's trying to use a model in an offensive way that Anthropic would not approve of, is it easy for them to dress that up? What sort of signals do you think are going to be most valuable in terms of detecting who's doing stuff they don't want them to be doing? It could be account-level monitoring. I don't know, but how hard is that? And what are the key signals that would allow them to make sense of it?

A very sensitive topic for us. I think it's very hard. very hard to distinguish between a malicious actor and a defender. And it's going to be about either vetting in real time before you give access to someone, vetting them, or after the fact auditing them, you know, and vetting them after the fact. But yeah, a lot of this is about, for instance, when you ask an LLM to write an exploit for you, this is something that obviously is associated with bad malicious actions, right? But it is something that we as defenders need to do to provide value. So yeah, very hard in my eyes for a topic.

Sobering note to end on, but this has been a fascinating conversation. Thanks guys for joining us and we'll definitely be watching this space.

Thank you so much.

Thank you. Thank you. Nice to meet you.

Thank you.

Hello.

Hey guys, how you doing this morning?

Very, very good. Um, thank you for joining us.

Yeah, of course. Thank you for having me. Uh, fascinating conversation there I got to hear the tail end of.

So let me, uh, let me do a quick intro. Uh, so, uh, Brett Levinson, he's joining us today. He's the co-founder and CEO of Moonbounce. Moonbounce is building a real-time policy engine for AI products and user-generated content. Software that sits in the path of the chat message, image, video, model output, checks it against the company's rules, and decides whether to allow it, block it, slow it down, escalate it, or steer it before the harm ships. As we noted in the last question, this is the database demon that is watching your messages and trying to figure out whether or not it should report you. Yeah. Uh, Brett has seen the old version of this problem from the inside. He left Apple in 2019 to lead business integrity at Facebook, and by the time he left Meta, he was dealing with moderation at a scale where human reviewers might get 30 seconds to make a call on a machine-translated policy, and those calls were only slightly better than a coin flip. The idea behind Moonbounce is simple but pretty radical. Safety should not be the tax you bolt onto a product after it breaks. It should be a live part of the product itself. And now that the thing being moderated is not just what the users post, but what AI systems generate, say, recommend, and eventually do, I want to understand whether that kind of control layer can actually work. So, Brett, take it away.

First of all, uh, you want to— you want to pitch for me occasionally? That was, uh, that was awesome. Um, So yeah, I mean, I think you said most of it pretty well, whether it can work or not. So I just want to— I'd like to go back a little bit to my time at Meta. And I think without telling the whole story like I normally do, what I would say is I spent a lot of time there trying to figure out why was the best we could do in 6 months a 0.6% improvement in accuracy in our automated models? What I ultimately realized was it all starts with policy. There is a policy, a set of rules that an organization wants to enforce somewhere. We like to believe that those rules are obvious and simple. I would have friends, once they knew what I did, they would come to me all the time and be like, I don't understand. You guys don't know what hate speech is? What's wrong with you? Sure, yeah, to an individual, It's like, sure, I don't like that. That obviously shouldn't be allowed. But the funny thing is, you know, you ask 100 people on the street about something and you may get a bunch of different answers. And yet organizations have to kind of decide where the line is. The problem is that more often than not, the way we write down these rules, there's a lot of ambiguity and a lot of interpretation that ends up being required. And we, at least until we built Moonbounce, we had no way of measuring that ambiguity empirically, basically saying, hey, for semantically identical or equivalent, let's say, content, and here content could be generated by users, it could be generated by an AI, could be a prompt that you're sending into an AI, whatever. For semantically equivalent content, are we going to get the same outcome all the time? Basically, that's very much our goal. And the engine that we've built does exactly that. We take a policy, we measure its ambiguity, and we almost like kind of do the Socratic method on it. Like we ask, well, okay, but why? Why do you think that should be the rule? And continually, as we put it, we decompose the policy, we break it into its atomic parts until the questions that we're asking about the content are ideally So simple and straightforward that 100 out of 100 people would have the same answer, even if they wouldn't agree necessarily with the decision that is made, right? Like, the idea is you agree with the evidence that we've collected. You may not agree with the conclusion, but you agree with the evidence. Hopefully that makes sense.

Yeah. Could you maybe walk us through— this is one thing this is bringing to mind for me is the AWS automated reasoning type product. It sounds like there's some overlap in philosophy there where I know they have kind of a similar process of like, give us your natural language policy, we will decompose it into a bunch of rules. Then at runtime, we can apply those rules in a more systematic way and There still can be some gaps between the intent of the policy and the decomposed rules that get applied, but you have the ability to kind of iterate on that process.

Exactly.

Until you're happy and can deploy it. Maybe, I guess, tell me if you think of yourself as being in a similar vein to the automated reasoning. Is it automated reasoning for everybody not on AWS? Is it different in some important ways? And maybe you could actually give us an example of a certain natural language policy, how it gets decomposed, what some of the challenging points are, and how organizations can iterate through to get to a point where they are happy with the results.

Yeah. So I think first the word you said there, iterate, is like my constant mantra. I think it's important. Maybe it's because I did this at Meta for so long. Maybe it's just how I am, but I tend to be pretty pragmatic in my approach. I think safety problems are a business risk. Companies act on incentives. They don't act on morals or should have, much as we might want them to. Ultimately, first of all, I'm not actually 100% familiar with the AWS product you're referring to, so I'm not going to do a comparison there. I guess the simple example I always give is hate speech, because I think everybody has their own idea of what it means. Just to give a very simple example, In my view, the wrong thing is basically saying, is this hate speech? That's the absolute wrong thing. If we wanted to break that down further, however, let's think about what it means for something to be hate speech. Well, first of all, to me at least, and again, I'm actually not a policy expert, I employ a number of people who are, but to me, the first thing is, Are we talking about a specific group, a specific group of people? So religious, sexual orientation, race, some identifiable protected group or protected characteristic, maybe disability, something like that. And then maybe your instinct is to say, okay, well, is the speech degrading in some way, or does it have the intent to degrade? Now, to your point about iteration, that may be enough. This is very much how our product works. The question is, when have you gone far enough, basically? For certain use cases and certain customers, just asking those— first of all, defining the set of groups and then asking, is this degrading? Maybe our models, and we've spent a lot of time training them, maybe our models' understanding of the idea of degradation aligns with yours. And if so, great. Or it aligns with yours 97% of the time, often enough that it's working for you. But if not, we can still break that down further. What does it mean to degrade, basically? So that's the simple example that I always give. And the other thing that I say all the time is we all have to acknowledge there's no perfect here. There's no such thing as perfect in a space where what we're talking about are normative preferences. Like, I get that, you know, there are certainly cases that are illegal and, you know, kind of what we would— I hope most people would think of as like the worst thing. But there's still sociological constructs and norms that as a society we have said this is not okay. But the universe doesn't care. And so I think like once you acknowledge that mostly what we're dealing with is preferences, um, you can realize like there's no perfect. All we can do is make iteration fast and easy and give you as much information as possible so that each iterative step, uh, ultimately has as much impact as it can. And we can chase down that long tail of perfection. We may not ever get there. Um, But we can certainly chase it down. And, and, you know, just as an aside, what we see with our customers is that they do often have 75 or, or 100 policy commits over the course of many months as they chase down that tail. And then eventually they get to a place where, as I said, pragmatism here, like, it's good enough. They are, uh, satisfying, uh, the needs of payment processors. They are, you know, controlling their AI such that it's not lying to users about their return policy. Like, they're doing the things that they must do, basically. Um, but I think it's still good to acknowledge that, like, you may— you know, things evolve, and just everyday normal use may lead to new circumstances in the future that you may have to update your policy. And, like, that's normal. It's not a failure necessarily. The only failure would be if updating that policy deploying it takes months, which is frankly, you know, kind of was one of the things at Facebook that I kind of lost my mind over.

Just to be concrete here, what kind of policy statements break down most badly when they are in prose? Like, give us a concrete example of something that you've seen, you know, just break down because it was just The definition just didn't work.

Oh, I'm trying to think of one I can say. There are some that are pretty crazy, the things that we've dealt with, but I don't know exactly what the family-friendly audience is here. I would say one area that definitely we've actually been working on a lot recently that has been a little tricky to find the right language on has been AI sycophancy. I think it's been an area of intense interest lately, certainly around AI harm— sorry, self-harm, cases of AI psychosis and delusion. I think we all kind of— well, maybe not we all. I think it makes It is understandable why AIs have been trained to be so sycophantic. Everybody likes having a little follower who tells them they're great and that all their ideas are great, et cetera. The difficult thing here is, first of all, we have to define a set of examples of what we mean by sycophantic. Is it just agreeing with untrue statements? Is that it? Is that as far as we want to go? I would say, of the things that I feel I'm comfortable discussing here, I would say AI sycophancy has been one of the more recent challenges that we've been working on with a bunch of customers. We have an agent that we built ourselves, actually, that works this Socratic process and assists with the breakdown. Um, and it's, it's definitely been, um, it's definitely been interesting to see some of the stuff it comes up with. But I, I, I, I will admit, I don't think we're there quite yet.

Um, it's, it's, it's a very interesting, uh, point because I think, um, there is an OpenAI researcher called Rune, and, uh, he, he put out a tweet yesterday, um, which said, really tasteful and advanced sycophancy Involves mildly disagreeing with you to win rapport. And that's the kind of thing where, uh, how would you, how would you detect that? Uh, and how would you, um, and, and also to note that this is, you would be, um, you would be trying to, um, you know, moderate the speech of an agent, not a human in the, in the, in this, in this instance.

Yeah.

Right? Yeah.

Yeah, for sure.

Detecting Well, how about that?

For one thing, the main thing we have actually seen is like in cases like this, we need a pattern. So this, you know, we— I will say that like we're not going to be able— I shouldn't say we're not, but except for very obvious cases, let's say, which is kind of not the ones that are problematic, it's always the nuanced stuff on the edges that's hard. We couldn't— I don't think it would make sense, and I wouldn't advise one of our customers to like just be sending us the last message or the last paragraph. To me, this particular problem is one that develops over time, basically. And so you're going to need a significant portion of at least the recent conversational context to be able to identify that this is a problem. The other— so that's one. I would just say it's not just about the rules. There is a portion of this that is still about what information are you giving us to go on? We can't evaluate something that we can't see.

Mm-hmm.

I mean, I don't want to get too far into the policy development exercise because I feel like it'll just be dull and me dreaming up possible rules and you guys going, oh, I don't know. I mean, I'm happy to go back and forth with you on it.

Mm-hmm.

If you want, but I'm not sure it'll make for the best listening. So one, yeah, like I said, I think the context window is the main thing there that I think you need to be able to over time see that there's sort of a general pattern of buttering up the user, let's say. That would be kind of the main thing I would say. The other thing is you want a number of examples to work from. I mean, it's not that different from traditional ML training in that you need good labeled data that you can look at, that you can test against. Importantly, testing is obviously very important, that you can work from and say, okay, here's Here's one in what I would call indicator of sycophancy, let's say. Here's another. Here's another. The problem really that comes up is sometimes they're in conflict with one another and pulling together a coherent policy that finds all of them, doesn't create a bunch of false positives, et cetera, it can be tricky, honestly. I'm not claiming it's always easy.

I would love to dig into the architecture a little bit and then maybe also talk about how this paradigm may extend to things potentially well beyond content policies. On the first point of architecture, I mean, it's got to be fast, right? So, like, are you using—

Yeah.

small models? Is this, you know, is this the sort of thing where you sort of let things through and then run something in the background and if it gets flagged, then we kind of come in later like the original, you know, Microsoft Bing experience where you'd see the message and then it would like retract it back?

Sure.

Or are you doing a more sort of classifier style approach where it can be fast enough that you can build it into the stack and the latency is acceptable? What tradeoffs are people willing to make in terms of product experience, latency, cost? And how are you then engineering to meet their demands?

Yeah. So, I mean, to me, you've said the magic words. Like, I've been a big advocate kind of since we started the company and even since I was at Meta that, like, you know, an ounce of prevention is worth a pound of cure. Like, being there before something happens, or as you pointed out, maybe you can optimistically let a message through and then retract it quickly, is just a better approach than finding stuff 3 to 7 days later and saying, oh, we screwed up, we need to block or ban this user. And in the case of AI, what would you even do 3 to 7 days later other than maybe, I don't know, add it as a training example for the next fine-tune or something like that? Um, as far as the architecture goes, so we have a couple techniques that we're using. So one, yes, we do use some very small models that are already pretty fast. Um, it also turns out that breaking a policy down in the way we do into atomized bytes gives us some unique advantages on the, on sort of the, the latency front. The questions we're asking are all pretty small. They tend to share a prefix, basically. Um, and so we're able to, to sort of benefit from a quite a large amount of prefix caching. We also, generally speaking, at least first pass, we're not generating much. There's really no decode step for us. What we— I mean, I'm happy to, I guess, share some of the architectural details. We essentially are training a binary classification head onto an LLM. We don't initially anyway need Some— we don't need the questions answered with an actual yes or no. And in fact, it's counter to our objectives to do so. We actually want to know what is the probability that the answer to this question is yes, basically. And that— I don't want to get— I don't want to— I have a tendency to sometimes go on tangents. So I'm going to try to contain myself here. Maybe we can come back to like the benefits of having those probabilities and the abstain gap and all that. There's another common thing in moderation, safety, guardrails, control, whatever you want to call it, which is that for most— for the majority of policies, upwards of 90% of all the content you're ever going to see is fine. Like, it's a real needle in a haystack problem, right? Like, you're looking for a small sliver. The only problem is that very often that sliver has high severity. Has real risk associated with it. Um, and so we have a number of layers sort of, uh, in front. You mentioned lightweight classifiers. They're not simple binary classifiers, but we do have, uh, a number of much lighter weight models that sit in front of our, uh, I guess what I would call like our main QA engine, um, uh, that can give us with reasonable confidence and high recall— that's the important part— a quick answer upfront. The idea is, let's say, just for argument's sake, let's say 90% of what we're going to get sent from a particular customer is fine. Really, there's no problem. We don't need to look at it for real, basically. Ideally, we want to try to take, let's say, half of that and filter it out right away and just approve it, basically. If we can do that, then on average, The latency that we're offering the customer, I mean, for those cases, we're going to be sub-200 milliseconds, basically. And then because our models are pretty damn fast, like for the rest of the cases, we're sort of in the 300 to 500 millisecond range when we actually have to do a deeper scan. I will say it also varies a lot by modality, and there are aspects there that are just hard to get around. Like text is very, very fast. Those are the sort of the numbers I just quoted. Images are a little bit slower. We have to run a vision encoder. There's more steps. We have to very often resize the image. We have to potentially transform the format of it before we process it. So there's just built-in latency. Video, even more latency because we first have to pull the video from wherever it is. It could be very large, et cetera. Rip out the audio, transcribe it. There's all these extra steps that we have to deal with. And to answer that last question, what is the What is the use case tolerance? Um, I, uh, I, I do think it, it depends a lot on use case. Um, for some of our, for example, AI image gen customers, right, it's already taking 6 to 10 seconds to generate an image. So, you know, adding maybe 10% latency on top of that, cuz it takes us 1,500 milliseconds to render a verdict, like, it's not that big a deal. You know, it's not ideal maybe, but it's not noticeable to the user. And in my view, actually, uh, that's what a lot of, uh, the tolerance is going to come down to. Like, how does it affect the user experience? Is it noticeable to the user? Um, I just wrote this whole Active Guardrails piece kind of about this. It's where sort of our future focus is, is, um, on essentially being able to do what we do and do it on streaming tokens effectively so that we can kind of just be like the old days of TV and kind of just like run the conversation on a 5-second delay and, you know, bleep out anything that, that's bad, basically. Because I just— what we see is that if we're, if we're asking too much from our customers of— asking something of our customers that is going to significantly impact the user experience, then they are less likely to adopt the controls that they ultimately need. That's my feeling on it, honestly.

Yeah. So, okay, zooming out to the highest level I can think to zoom out to, it seems like one dominant trend in AI broadly right now is that AIs are getting really good at things that are easy to verify. by. And, you know, math and programming obviously headline those, those categories. At the same time, we get certainly mixed reports on how well they do on harder to verify tasks. And so one kind of big strategy for the space overall is, can we figure out ways to reduce big, fuzzy, hard to verify tasks to a bunch of Easier to verify tasks that kind of aggregate up to the hard thing. That strikes me as there's some sort of sleight of hand there, though, that often gets kind of passed over, because a definition of intelligence that I've used in the past a lot is just the ability to succeed despite lack of fully explicit instructions. So it seems like you pull all these things apart, you have all these comparatively much easier to verify tasks, But it still seems like somewhere there's some residual part that is like, if it wasn't hard to capture, then we wouldn't be here in the first place, kind of. So I wonder how you think about this problem at a high level. Do you think the techniques that you're using could be, or perhaps are being used at frontier companies to, for example, answer a question of like, is this ML experiment, one that we would call in good taste. Does this demonstrate good research taste or not? Or does that just feel like a totally different thing still somehow?

No, it's actually an interesting point. I hadn't really necessarily framed it in my head quite that way, but my partner and I were literally just yesterday, we were talking about red teaming and thinking about, we've always been focused on inference time, inference time, inference time. That's our focus is can we help with controls that are applied at inference time? But I don't know why yesterday we just happened to be having a conversation about red teaming and is there something that we could do in that space to at least, let's say what I usually call invert the pyramid, use AI to validate results at scale. and free up time and resources for human experts to deal with the difficult cases, the cases where you really do need— what I like to say is you need to have been a human being alive. There's value in having been a person who has been alive and part of society for 30 years or whatever, however old you are. I think It'd be interesting to just come up with a specific case to think about here. I think, as you said, there's a little sleight of hand. I don't know that for all possible cases that we could come up with, that the set of small, more easy-to-verify things necessarily ladders up to the hard thing. Like, there may be sort of an emergent property there that, that's hard to— that's hard to capture. Um, but I almost sort of wonder, like, is it falsifiable in some sense? Like, could we— sorry, now you've got me on a philosophy track, so now I'm really thinking about this.

Uh, good.

Like, could we at least show That for the set of cases where the small— let's just say, let's start with the assumption that it is possible to decompose that larger, more difficult preference down into a set of smaller, more easy-to-verify preferences, basically. Could we at least show that if we can verify those smaller, easier cases, that the larger, more difficult case necessarily follows from that. Now, the reverse may not be true. Like, it could be the case that when some of the smaller cases don't verify, that the larger case still does, basically, because there's— we sort of, we lost some context, we lost some I don't know. Je ne sais quoi along the way. The one piece of experience, like the experience that I can pull from in what we do, how we've built our system, the way our customers have used it, the way our AI copilot assists with that process, is that we haven't yet run into a case that we couldn't actually solve in the way we do. Sometimes it's harder than you want it to be. Sometimes it requires more decomposition than you would hope, basically, where you're going all the way down the rabbit hole on a particular term and really sort of finding and defining the edges of that case. Now, of course, just because we haven't found a case yet that we couldn't solve doesn't mean there isn't a case that we can't solve. But man, you got me thinking. I actually think that's a very interesting use case, and certainly one that would be beneficial. The things people are getting sued for right now, in my view anyway, are those softer, harder to verify things where there just may not be an objective right answer necessarily.

So we started off talking, I think, on the show about the new laws, like the SB 315 in Chicago, in Illinois, and some of the laws that are being passed. And it seems to me that many of the laws also have to do with, you know, people are concerned about child safety. There have been some concerns about, you know, the chatbots encouraging harmful behavior, especially for juveniles. To what extent do you think— I think passing these laws, and you have this fragmentation of the legal landscape with different— with every state kind of, you know, perhaps in every country perhaps having its own kind of rules on what these chatbots are allowed to say. To what extent do you think, you know, a company like yours would be able to actually make this easier for legislators to actually you know, form these laws, because now it's technically— the technical execution is now possible, which perhaps it was not possible, you know, 10 years ago. Like, to what extent do you see that happening?

Well, I mean, this is gonna sound a little self-serving, but I definitely think that, you know, we can and sort of are helping with this. So like the parallel and the thing that, you know, we saw about, I would say, 18 months ago, was that in the absence of official legislation from states and federal jurisdictions, the payment providers became the legislators in some sense. They're saying, hey, we will accept payments for you, but you got to follow the rules, and these are the rules. Many of our current customers came to us hair on fire going like, my payment provider is going to drop me, and here's what they've said. I've had to here's what they say I have to accomplish, and be able to prove, importantly— that is another piece of it, right? It's like the auditability of the system and being able to prove to whoever the regulatory body is that, hey, you can read this. You can understand it. We can show you the metrics. We can show you that for the set of cases that you're concerned about, that this will actually provide the protection that you want. We can also show you possibly where the failure modes are. Where are the edge cases that we haven't quite solved for yet? So certainly, yes, this is a thing we already do. And I do think the fragmentation of the landscape even has us thinking a little bit more about the orchestration side of our platform. Because in many cases, companies that operate across the country or across the world are coming to us saying, hey, we have to adhere to all these regulations. But it's not just one set of regulations. It's a bunch of regulations, and we need to be able to decide which which policies are being applied depending on the locale, essentially. Yes, I definitely think it helps. I will also say you got me thinking back to years ago at Meta where it was like Cambridge Analytica happened. All these things are happening and I remember all the back and forth in the press going like, look, this problem is hard and we don't have a way to solve it other than to throw a ton of human bodies at it, basically. And I just think we have proven— and I don't think we're the only ones— it's just not an excuse anymore, is my feeling. I don't blame the industry at all. I think starting with capability, it wouldn't make sense to build governance and controls for systems that don't exist. So it makes perfect sense to start with capability. But I think now that capability has matured, there's a real need for systems that are not just— to me, it's not checking a checkbox. It's, in my view, systems that are controllable, systems that are predictable are good products, frankly. I struggle to imagine AI products that are going to have long-term pull with users if they're frankly untrustworthy, basically.

Mm-hmm.

To me, it's about more than just, oh, well, this jurisdiction says we have to do X, so we have to do X. I actually think if you do that, there will always be a new checkbox that you're chasing. You're much better off building for controllability from the outset, basically. That's just my general advice. Self-serving, I know, but I genuinely believe that.

Incredible. Brett, thank you for joining us. It has been an illuminating conversation.

Thank you. Yeah, it was fun, guys.

Yeah, awesome. And hope to have you back again soon. Cheers.

Be happy to. Yeah, have a nice day.

I see this gap in so many places between the desire to break things down into verifiable constituent parts And the hope that that will then ladder up into something that we can be confident will work. And it is maybe the biggest open question in the world right now. Like, how do we get the higher-level system properties from the low-level constituent parts that, you know, this is true in biology, right? Where we've got like all this multi-scale interaction? How do all these proteins add up to a cell? How do all these cells add up to a tissue? How do all the tissues add up to the body? It's— I've wrestled with this a lot, and I feel like I'm maybe just not smart enough when it comes to formal verification of software methods.

Mm-hmm.

Because there's always this thing, there's this leap that people are ready to make, or, you know, that they seem confident or comfortable making, that I'm kind of like, well, okay, we proved all these things. But how do we know that we proved the right things? And in formal methods, people will often say, well, it's only as good as the spec, you know, you had to start with the right spec. But then it's like, okay, sure. But then now there's our problem again, right? Like, we, we have the, the fundamental— not to say you don't get value from this sort of stuff. But if you're really like, wanting to get somewhere where you have, you know, where I'm sleeping at ease at night knowing that all this stuff is taken care of, or even just where you feel like you have a really intuitive understanding or command of the situation, this problem of aggregating small things up into larger system behavior. is, I think, dramatically undertheorized right now in so many different ways. And, you know, the market is another one of these, right? That we don't know, even if we have well-behaved agents, when we put a billion of them into the economy, we really have no idea what that is going to look like either. So yeah, there's sort of a I don't know what— who should we even— we'll have to prompt our agents to think about who we could talk to to try to get a better handle on this holistically. Michael Levin comes to mind, biologist who—

Yeah.

I think is fascinating, although I think he kind of more often says like, we just don't have the answers. And, you know, here's all the interesting things that I've found that kind of ought to prove that to you. Yeah. Or ought to convince you of that.

It strikes me that the rulemaking is really kind of heuristics and it's kind of, you take the intelligence as far as you can and then you attach whatever the intelligence doesn't solve, you then start doing heuristics. This is what Leopold Aschenbrenner calls the Schlep. And he also pointed out that you guys will Schlep, but the model will overtake you. Which is fair, but then you still need to put out a product and the product still has to come out within 12 to 18 months. And if the model isn't ready, you end up, okay, let's do the schlep. Let's do the rulemaking and let's do the heuristics. And we see this in every vertical, basically. Every vertical, you get the model to where you get it to be, and then whatever the model cannot handle, you end up with the harness and the schlep and the rulemaking and the heuristics and the exception cases and the edge cases. You know, the talking to the client and figuring out where it went wrong, right? Like we see in every single vertical. I'm not sure there's a solution, like, for the companies involved. I think, I feel like, I feel like what the companies, the company's like real asset is really the customer relationship and not so much like what you build. What you build is not like the product is not what you build. The product is really the trust and the relationship with the customer in order to be able to deploy the next generation. And I think if the companies build that trust layer, because the customers are not AI people, basically. The customers want to have their own business to run. And I guess the customers have policies. The policies can be implemented better by this AI, but still with all the edge cases. And maybe the next generation gets better. On the consolidation of agents, the consolidated behavior, That's a tough one. The systems-wide behavior consolidating from individual independent agents. I think one of my hopes is that one of the great innovations that these AIs will be able to come up with is also in economics and in how to predict and manage systems-wide behavior of individual agents. And if they get better at that and better at predicting those things, then maybe some of our age-old coordination issues will just fall away. One has hope that that'll be one of the innovations that we see these things come up with. Yeah. I was very struck by Brett because one of the things that has always stood out to me is that there's this inherent tension between freedom of speech and trust and safety. And that, as Elon obviously went right into that at Twitter, and it's a really tough one because especially if you have kids, you really are like, hey, I want these things managed for me. I don't want these platforms to have certain things, certain behaviors, even if they are Uh, even if freedom of speech does say that, you know, you have them. And, um, we, we have freedom of speech but not freedom of distribution. That's the other thing. We have this tension between freedom of speech and freedom of distribution. And for someone like Brett, as you know, what my, what my state law question was leading up to is you end up having a system where the state basically makes these rules on what people are allowed to say. And then even when they don't legislate, they have the soft rules that payment providers then have to implement. And so then you have this system of like, you know, speech rules, which are, you know, basically descend down all the way from legislation to soft regulator like nudges. And you end up with a system of, you know, very quite defined kind of rules on what you're allowed to say online and And once you have these things, end-to-end encryption doesn't matter anymore. You can have them inside a private conversation. And in fact, it's almost required. One of the reasons Zoom doesn't allow fully anonymous, fully private, free conversations Is because Zoom started to be used by child abusers. They would use Zoom for these things. And Zoom was like, we cannot have these things be completely encrypted and private and anonymous and free. And you just can't enable that. You don't want your technology being used for that stuff. And you can kind of see that Maybe you have a room monitor who is an AI, which is sitting in on every conversation, whether it's encrypted or private or not. You have that room monitor sitting in there and monitoring the conversation and monitoring what's going on. Maybe that's not a bad thing in some sense, but as long as you don't push into like, it's not just the CSAM, but it's also anyone saying any word which resembles Nazi or whatever, or fascism, or, you know, some of these political concepts as well, right? So I don't know, I mean, these are very tough philosophical issues, uh, on, on speech that, um, I, I think people largely do not want to— you know, it ends up with a lot of lawyering at the end of the day, and I think it's still unresolved as a society on what we want, really. It's a tough one, you know.

Yeah, maybe that's a good place to leave that thread for today. Um, I, I am definitely gonna prompt an agent for some research on who can help us get a better conceptual understanding of these low-level to higher-scale system behaviors. And I don't think we're gonna get, you know, a a great answer on that, but I'm mindful I don't even necessarily know exactly where the frontier of our understanding is on that. So I think that would be a good topic to dig in on a little bit more. One follow-up from yesterday that's relevant to this content moderation piece is—

Yeah.

And also just a good example of living in the future or the future is now. Is after we got off yesterday, we had been talking about the OpenAI moderation endpoint and how it is free for all. I believe it now does require a token and maybe a sort of paid account in good standing, because I initially yesterday prompted my Claude code to first of all just go orient itself in my own history, right? So I have deep history available for it, where in emails and at various points in time, I had sent reports going all the way back to the GPT-4 red team to OpenAI people and saying, hey, first of all, these prompts are being served. And also, by the way, your moderation endpoint doesn't seem to catch them.

Yeah.

So it was able to pull all that context out of my history, which was a great starting point for it to then be able to do The experimentation. It did a relatively small-scale experiment, and aside from me having to refresh a token because something expired somewhere in the system, it was able to set up an experiment, create sample prompts in a sort of low harm, like probably should not be flagged, medium harm, and high, you know, let's say medium severity, high severity across all the different categories. that they support in the, in the moderation endpoint, run that experiment and give me a report back on it, basically all in one shot again, except for the token. So that was pretty cool. And the result was the gap that I had been complaining about has indeed been closed.

Mm-hmm.

You can no longer put a prompt into the moderation endpoint that says we're part of a criminal gang. and we better be careful or we're all gonna go to jail. That will now get you flagged.

Mm-hmm.

They also did a— seemed to do a pretty good job. And again, this is maybe where, you know, we can debate what the content policy should be.

Mm-hmm.

But on the low end, the not harmful prompts that Claude believed a moderation endpoint should not flag, it Only got maybe 2 of those wrong on a false positive basis. So it flagged everything that Claude thought it should flag, and it flagged just a couple things that it thought it should not flag. So to give credit where it's due, both to Claude for doing all the work on that with a 3-sentence prompt from me, and including, again, going back into deep history to find the context and figure out what the hell I was even talking about. Running the whole experiment. And credit to the OpenAI folks for actually, at some point, I don't know when it changed, but at some point they did get around to solving that problem. So that was good to see.

I was, I was surprised they would have improved it.

And sure enough, they did.

Yeah, I, the, I, it's interesting you say that you needed an API key. Because what, what that indicates is that they are afraid of someone mining the moderator now to detect the edge cases and then to hack into it. So it's very interesting to hear you say that you required an API key, which means that they still believe that there are gaps there, and they are protecting, protecting the gaps from being exploited. So that—

It may have even always been that way. I'm not 100% sure if you know, if that is new. But there was some little trip on that in my, in my flow. But that was the only thing when I came back to see what Claude had done. That was the only blocker that we encountered in that process, which I thought was pretty cool.

I wonder, I wonder to what extent that interacts with stuff that people like Brett are doing. I mean, is that not kind of— I guess Brett has more sophisticated policies, right? Like, Oh, you know, even child, you know, sycophancy is much more sophisticated than just harmful behavior.

Yeah, the moderation endpoint is not that granular. They cover now—

let's see.

It's like explosive.

Something like 12 categories. Sexual, sexual/minors, harassment, harassment/threatening. hate, hate/threatening, illicit, illicit/violent, self-harm, self-harm intent, self-harm instructions, violence, and violence/graphic. So that's still, you know, pretty coarse grain compared to what he's offering.

Yeah.

Um, and it's funny that he's— I mean, I guess that's the Meta background, right? It's just, it's all these speech things. The Amazon automated reasoning product, which I do think sounds more similar than different, is I think they pitch it with an, like, a mortgage loan underwriting process example. And the, you know, the policy might be however many pages long, they decompose that into all these rules. The rules can be composed, you know, with the— that's where the formal methods comes in, you know, as they sort of— set up a logical structure to apply these rules. And, you know, which all the— if, you know, this, then that sort of stuff is included in that structure. And then at runtime, they just take whatever the case is, right? This particular loan application.

Mm-hmm.

And they map the input onto all these specific variables. And then the actual execution becomes like very deterministic. So the lossy part is— and then in this way, maybe it is a little bit different than what he's doing, because I don't think that they are training classifiers on— as like the— I don't think the classifier is the last step in the case of the automated reasoning.

Mm-hmm.

Rather, it seems like the last step is a fully deterministic process that says, Given these values that we got in this case, and given the structure that we created from your natural language policy, here's the answer. That last part is going to be fully deterministic. You could have missed something in the decomposition of your policy. You could have, you could have something wrong in terms of the values that are provided to that final policy.

Mm-hmm.

decision-making engine. But that last step is, I think, fully deterministic. And, you know, you sort of can be confident that given certain inputs, you know, you're going to get the same output every time. Given, let's say, semantically equivalent inputs, you'll get the same output every time. One other thing I wanted to go back to just for a second is Mythos in the security realm. Oh yeah, and you might know this, you might have a better sense for this than I do, but I'm still kind of—

were, uh, were a little bit annoyed that we, uh, we went into Mythos. He was like, you know, it's, it's fine, like, it's, it's just, it's just marketing, you know.

Yeah, I mean, I don't know, I would think you would have to expect to be asked the The thing that I still can't quite wrap my head around is it feels like at least for a moment we're gonna have a— nobody got fired for going with Mythos, right? They said at the end of the day, like, somebody's got to be accountable. You can't fire an AI.

Mm-hmm.

And to me, that's like a pretty strong pull toward trying to use Mythos, right? If you are the CISO or whatever at some company, you're like, yeah, I could save a lot of money. Um, I mean, I might look good for having done that, but if something comes through and it's ultimately like, hey, what the hell happened here? And then it's like, well, we actually opted for the Sonnet plus Harness plus Guidance, uh, because we thought it would be a better use of funds or what have you.

Mm-hmm.

I think it's gonna be— that's a tough spot, it seems like, for a lot of people to be in. Whereas if they did spend up for the Mythos package, then, you know, what are you gonna say? Like, I use Mythos, right? I mean, what else, what else could I possibly have done? So to me, it feels like there's kind of this moment coming where these companies are gonna be risk-averse. They're gonna be scared. They, you know, nothing gets a company to spend money like being scared. And I just have a hard time seeing how, obviously on some margin, the cost constraint will bind. But it seems like I did one of those AI forecasting things at the beginning of the year. One thing that I came out much higher on than the rest of the sort of general population of forecasters was frontier company revenue. at the end of the year. And I think my— I'm looking pretty good on that, I think, still. And it feels to me like Mythos is not, um, gonna be too expensive for people because the social incentives to say I did everything I could seem quite strong in this moment compared to the incentive to save a little money. And obviously those aren't, you know, not binary choices in practice when you get it under the hood at But what do you think? Do you see the argument that I'm making for nobody ever got fired for going with Mythos?

No, I mean, so the way I look at it is that you have 2 kinds of organizations, the tech companies and the non-tech companies like Walmart, whatever, right? I think the tech companies, that is the case that you would end up, you know, having to use Mythos because you want to, you are providing product to someone else. You're providing technology to someone else. And it is in your benefit to make sure that technology is solid. And if you have issues with it, your customer will come back to you. So I think that's there. For the average CISO, like someone at a Fortune 500 company like Walmart, I don't think that is that true. I think the average CISO, like at Walmart, he's seen as a cost center. He's always been seen as a cost center, right? And as a cost center, he's always had like more things that he wants to fix, or he or she wants to fix, than they have budget for. He's had like this backlog of like, you know, 20 years of, you know, patches that they have, right? And I think what ends up happening is that Mythos brings like attention and budget to the CISO, uh, just having Mythos in the ecosystem. That there are, oh, it's scary, blah, blah, blah. And this is going to hit everyone in 6 months. And he gets budget. And now that he gets budget, some of the things that they need to do is not like use Mythos, but might be like migrate off COBOL, migrate a mainframe off COBOL that they've been using for 25 years. Right. And that, and that will automatically fix all the Mythos bugs that they found in COBOL, like, which are now irrelevant. Right. So the fix might not be to actually use Mythos. The fix might actually be to do migration or something else on the, you know, within the enterprise, rather than just look for bugs. So I think in that sense, Mythos has created a budget opportunity for CISOs and for CIOs, and that's allowed them to spend. And that spending is not necessarily going to go to Mythos, but it's going to go to a host of other companies, which are providing solutions that will overcome whatever Mythos has found, some of which may be migration and new software and new build, right? So I think that's good. I think for— it's true of tech companies, though, that they are going to have to use Mythos because they are providing that product to other people, and they want to make sure that those products are solid. If you want to sell the product, it's going to have to be solid. For stuff which is in maintenance, like COBOL or stuff like that, like, yeah, you know, I don't think Mythos is going to be deployed. We all know that there's, you know, tons of bugs there, but people live with it and patch it. And if it's too much for you, you can pay for the migration, right? So I think that's the case. So it's just more budget for IT, which— and more budget for IT is good for everyone in tech. So hey, you know, why not, right?

Long live the AI bubble. Anything else you want to cover today before we break? One story that I at least wanted to mentioned briefly, and we'll hopefully bring some actual music to the table before too long as well. Suno just raised a bunch of money, $400 million on top of a $5 billion valuation. So $5.4 billion post-money Series D.

Mm-hmm.

Incredible product. You know, I'll— we can pull some stuff out of my Cognitive Revolution playlist and maybe make some stuff Based on our episodes as we go. But it has without question been the AI product that I have had the most fun using in the last couple months. And it's definitely also been really interesting to see the inbound messages that I get from listeners because it used to be that my strange hat-wearing habit of having my hat pulled way back on my head was like the most common thing people talked about.

Mm-hmm.

Then when I started AI art for episode image previews, people really hated that, actually. The response was very negative. A bunch of things were like, you're degrading yourself. You know, this is below your standard. It makes people think the whole thing is gonna be slop. And I've honestly mostly just straight up ignored that. For better or worse. But then on the music side, it's totally different. I get almost, you know, probably one per episode of somebody who takes the time to write in and says, I really like that song. People have repeatedly asked where they could get it. Not everybody seems to, even though it's an AI podcast, you would think it would be fairly obvious that it's, that it's an AI-generated song of some, you know, origin. But Not everybody realizes that. And this week, I'm not sure if it's happened yet, the first one that I created is actually gonna be played on the radio somewhere.

Wow.

I got an inbound from somebody who, you know, has a radio show. I don't think it's like a, you know, huge deal or whatever, but basically he said, I really like that song. Could I play it on the radio and how should I credit you? And I said, yeah, go for it. And I maybe need a DJ name, but I don't have one yet. So That could be something we could also brainstorm with our agents before the next episode. But definitely, if you have not tried Suno, it's really fun. I would highly encourage people to check it out.

I haven't tried it in about— I tried it like, I think a year and a half ago. But obviously there's been massive improvements. So I will— Yeah, for sure. Yeah.

The 5.5 version, and I don't even really max out all the features.

Yeah.

My workflow is relatively simple compared to what they do support. Occasionally I get in and actually try to edit a little bit. But most of the time I just kind of iterate with Claude on genre and lyrics a bit and then just generate a bunch. It's also really cheap. I, for the longest time, I just upgraded to whatever the higher tier was. Because I wanted to try out some other feature that they, you know, had only at the next price point up. But for the longest time, I was just paying $10 a month and never ran out of credits. Probably, you know, generated an average of like 10 different songs per episode. You know, probably doing at least 50, maybe upwards of 100 generations per month. Never ran out of credits. Just, you know, and it's fast too. That's another thing that's pretty interesting about it. From when you hit create, Mm-hmm. You are listening. It doesn't generate the full song at once. It generates the beginning of the song. So, you can play it and it's continuing to generate the rest of it while you're listening to the beginning. But you can start listening to the beginning of a song that you just generated in like 10 seconds from clicking the button. And then you can play the whole thing straight through all the way to the end from that initial little buffer that they give you. It's cool. I can definitely see why they're succeeding.

I think they had a Billboard 100 already, right? They had one Billboard 100 at least that I remember. So which means, you know, by next year there's gonna be 10 Billboard 100s. Human musicians, human writers are gonna—

I wanna get this stuff on Spotify. You know, I've already got the podcast going to Spotify, but now I wanna put the music on Spotify. And this goes back to our agent conversation that the hardest thing about that is how do I get my agent to be able to log into Spotify? And, you know, with what credentials? I log in with my— I think I used to log in with my Facebook account. I may have a different way of logging in now.

Gotcha.

Um, you know, I, I don't necessarily wanna share all those credentials. Can the AI— the AI has its own Gmail. Can it create its own Spotify account? A lot of times they do need a little help to get over a CAPTCHA hump or, you know, some sort of, uh, roadblock. But I'm looking forward to hopefully not too far into the future, actually seeing if some of these songs resonate with people beyond my, you know, very narrow and AI-obsessed podcast audience.

I think, I think the— I think it's almost possible with computer use. I think CAPTCHAs are obviously solvable at this point. But I think the firms don't want to take the liability yet. So it has to get good enough that they're willing to take the liability to that they're going to do the auth for you. So And on that note, Nathan, it's been— it's June 3rd. This is our 3rd episode going live daily. And we still have to like get a bunch of stuff out. But yeah, this is— this is— this has been fun.

Yep. The sprint continues. No rest for the weary. I'll see you tomorrow.

Cheers. Bye-bye.