AI:AM — Fable, AI Safety and Julius · June 10, 2026

the 2 hosts. Good morning, it is, uh, Wednesday, June 10th, 9:00 AM Pacific time. Welcome to AI AM. Nathan, hello.

Good morning, Prakash. It is a historic day and, uh, excited to try to make sense of it with you.

Indeed.

Fable is here, and I think, um, it really calls for a Pretty fundamental recalibration. There's so many different angles. Something like this is really a historic artifact, a historic threshold crossed, I think we're gonna see it as. And I think we should all kind of take a minute at least to reflect on like how it should change the way we're working, thinking, planning, planning horizons. Because I don't think it's gonna be their last jump of this scale either. It really just keeps working. So for me, Wow. the biggest thing so far has just been the quality of work that it does on the things that are most idiosyncratic to me. It really feels like not a perfect, by any means, write-as-me companion, But something that I'm kind of willing to become a hybrid with in a way that it hasn't risen to in the past.

I—

we've had a lot of reactions on the timeline over the last 24 hours or so since the drop. I think just to take it off the top, the first was obviously the benchmarks themselves. The benchmarks are basically state-of-the-art in multiple vertical categories, almost every category that they've chosen to disclose. One note on the benchmarks is that they have chosen to disclose the benchmarks with a fallback to Opus 4.8. So in every test, it tries Fable, And then it falls back to Opus 4.8 if Fable is not able to respond. I think this basically increases the benchmarks by about 2 or 3%, which is quite significant. And there are some complaints, I think, online that this is slightly misleading. It's also slightly misleading in the sense that the amount of compute used for answering these questions is effectively 1.5 times, kind of roughly. You have the entire Fable compute, and then when Fable decides not to answer the question, it falls back to Opus, and then you have the Opus compute as well. So you actually have additional compute being applied to these problems. Having said that, sometimes the researchers are very clear on what the benchmarks are, but the business team feels the, I think, economic pressure and the strategic pressure to show better benchmarks, which are really kind of a rough guide to what the model does. And so we saw this at Meta as well before. And so you see Anthropic falling prey to this. So you can definitely feel that they see that they have that pressure for performance right there. Besides that, we have been seeing a lot of interesting commentary on the timeline. Nathan, what has struck you so far from just other people's reactions that you've seen?

Well, the scale, you know, at Recursive not long ago, the consensus was that the meter graph is kind of hitting its end because we just can't create tasks this big. And I think the really striking pattern on, on the, you know, in terms of what has kind of broken through and caught people's attention is just the scope of it. So often, you know, these are clearly tasks that would take days or weeks worth of work. And the reliability, I think, is still kind of unknown. You know, obviously people are spinning these things up, like seeing that the demo works, being totally amazed. And there hasn't been time yet to find all the rough edges, I don't think. So clearly there's a little bit of a euphoria warm glow, but the scope is just vast, you know, to create these worlds with physics, with, you know, trees made of math, and to do it at this scale with this kind of ease is, I think, the most remarkable thing. It really does suggest that a different way of at least digital work is going to be possible. And that word pressure that you used, I think, is going to apply all over the place kind of now because it seems like there is going to be a pretty high price attached, at least relative to what people are used to with the $200 a month. You commented just before we came on that they managed to reduce the price substantially from the original Mythos price that they had posted, which I thought was a great observation and does show there's probably going to continue to be downward pressure on prices for a long time, just as there has been the relentless march of capabilities progress. The deflation, I don't really expect to end anytime soon either. Even so, this is going to be one of those things where a lot of people are starting to make judgments and feel like they kind of would be reckless, if nothing else, to, uh, to not economize at all. Um, and so everybody's gonna now be kind of, I think, a lot more conscious of when they're going into the true top model and how much the top model is delegating to the others. That's something I think we're all gonna start to watch, um, a lot more than we have. But I think it's gonna be worth it for a lot of people. Everybody's gonna want some. You know, I really do believe that this is, this is a model that unless you're locked out, you know, and there, there is so much complexity around the, the way they did this rollout. But if you're doing something that's, you know, far enough from the, the nerf zone that you feel confident that you're getting the best, I think most knowledge workers right now would be compelled that, you know, at least would be wise to make sure they're using some for their sort of top-level planning and delegation to sub-agents.

So one thing to note about the nerfing. So what has happened with Fable is we have a lot of rejections, and whenever Fable decides to reject you, it drops from Fable to Opus 4.8. So there's a natural downgrade. In experiments overnight, I tried to fix— make a number of bug fixes on this very Studio app. And what I found was Fable would always consistently drop to Opus 4.8 whenever it was asked to do anything in production. So touching the production database, touching any of the security keys, touching, you know, asking it to review production directly. In every case, I've had 3 or 4 times it's dropped out. Every time it's dropped out, I've basically restarted the conversation, added back the context that we were using, but excluding the parts about going into production or addressing the production database, and it has continued work. So I think there are a number of triggers there. Online, people are saying, hey, it's not going to do machine learning research for me. I think that's just tip of the iceberg. You're seeing that because the people who are testing it intensively right now are machine learning researchers. If you were, I think, to test it on finance or you're budgeting process and you told it you're going to be directly addressing my QuickBooks or Salesforce, I think you might see similar results. Fable right now, I would say, is a research release, almost a preview. It is there so that I think they can judge the demand because they don't have a sense of how intense the demand is going to be. And they're going to judge whether or not it's safe to release, which are the— they've started off with the most constrained version of it with the least number of functions which are open. And I think over the next few weeks, they will start to take away some of those gates. And as they take away some of those gates, I think we will see both the increase in usage And some decisions on what needs to be really gated and what doesn't. So I think we are in the early stages of exploring what Fable can do. What struck me, as I pointed out, is that they're decreasing prices by about 35% per month. Mythos preview was announced exactly about 2 months ago. And in that time, they've decreased prices by about 60%. If you continue to see that happen, you might see in 2 months or so, 2 to 3 months, you might see Fable prices drop to where GPT-5 or where Opus 4.8 is right now. At that point in time, you'd probably see the next version of Fable start to be launched. So I think it's safe to say that if you're willing to be a couple of months behind the frontier, still going to be economic for you. If you find that you have to be at the frontier all the time, then you are in the higher paying category and you will remain in the higher paying category. So it's really a question of where you choose to be, I think.

Oh, my gosh. Sorry, that's on me. On your guardrails point, they're going to dial it in. And I think I spoke to somebody at Anthropic yesterday, a friend who said, we did a better job with this release than I thought we would. And the roughness of these guardrails and that comment and just so many aspects of this reflect the fact that the race is absolutely as on as ever. And this is all very just in time. This is not the same exact model as the original Mythos. The pricing has changed. Optimization is everywhere. Optimization is being driven through Mythos-level auto research, right? I mean, recursive self-improvement is on the pricing page when it comes to presumably a lot of little mythos step-downs on the price optimization curve from where they were a couple months ago to now. And so, yeah, that model is a little new. The filters are a little new. The techniques are probably the first time that they've been deployed at this scale, certainly when it comes to nerfing through you know, feature detection and steering vectors. Um, this is really, you know, some, some science fiction, uh, stuff unto itself, you know, relative to a couple years ago. All these techniques— I'm old enough to remember when sparse autoencoders didn't exist, uh, and now we've got, you know, NeRF in production.

Yeah.

Based on that technology. So it's all going to get dialed in, I think, and probably pretty quickly. And most people, I think, will not be too bothered by this. But the, you know, the core political economy question might still be the ML research, as obviously that is like really where they're looking to take off on the rest of us here.

Yeah, I, I think we will see. I have, I have hopes that I think the OpenAI guys were congratulatory But I mean, you know, otherwise, I think they are ready with another release. And OpenAI also has access to a lot more compute than Anthropic has at this time. We've also seen the prices of compute increase. So what this means is that, you know, there's a little bit of a metric here. Google is paying SpaceX 6 times what CoreWeave got paid last year. So CoreWeave leased out to OpenAI and the pricing that Elon is leasing out to SpaceX is 6 times. So what this means is that that is really where the price of compute, the marginal The marginal GPU, the marginal Blackwell GPU is now 6 times more valuable on a per-hour basis than it was last year. And this also means that Anthropic is also paying 6 times basically what they would've paid if they had booked out the GPUs last year. So this is where Sam has, by committing early and by almost basically overcommitting last year to the extent that OpenAI would go bust if things did not happen as they are happening. Sam has secured a significant cost advantage, something like 88% cost advantage at this point. And so that is significant. And that is now going to allow him to expand capacity faster. So he has his own recursive self-improvement dynamic within the financing space where, because he has booked capacity at a lower price, he's then able to, again, in the next cycle, also book capacity at a lower price. So right now, Sam is, again, he's building out while Anthropic is leasing. So Anthropic gets the capacity immediately at 5 to 6 times the price that Sam will get the capacity next year at. And Sam has already fulfilled his capacity for this year. At 1/6 the price that Anthropic is buying at. So he has his own recursive self-improvement loop in the financing. And we might see this because even if Anthropic has a 2-month lead on the actual capabilities themselves, on deploy, they might be more than 2 months behind because they have spent all this money, haven't spent the money on the capacity. It's a little bit of a microcosm of what might happen between China and the US if China deploys much more quickly than the US and the US has more capability earlier, but is not able to deploy as quickly. So it's very interesting to watch this dynamic of the interplay between the financing and the capability come about at this point.

Yeah, I think another kind of meta-reflection on that is just, unfortunately, it does seem like close watching and close analysis of the top-end companies is really where a lot of us should be spending our time. If we want to understand where this is going and help steer it at all, The focal point seems to be, you know, really shrinking into a pretty small set of actors in some ways right now. And, you know, I love taking the broader view and the more sort of, you know, gonzo journalist view. And there's, there's so many fascinating aspects to AI as it's playing out across society. But these dynamics really are becoming pretty central. This is all happening at the same time, of course, that they're starting to talk about a coordinated slowdown and what that could potentially look like. And the relative— it's such a high-dimensional competitive space as they're vertically integrating from everything from raw materials, energy, you know, all the way to putting groceries in your, in your online grocery cart for you. It's, it's the longest vertical integration, you know, in history, that's for sure. And so they're competing at every layer of this stack. And it's sad. It's really sad that there's so much tied up in these kind of interpersonal dynamics. It's a really— it's a sad commentary that you've got all these guys that know each other for so long, you know, go way back, shared this vision, in many cases started working on it together.

Mm-hmm.

Fall out with each other one by one to varying degrees, you know, and Sam Altman has used this language himself. faced the corrupting power of AI and clearly not always been their best selves. And like, you know, they hate each other. It's really sad. You know, Altman or Elon hates Altman. Dario hates Altman. Like, that's really such a toxic environment for these decisions to be made. I think back to the the Dario and Sam inability to hold hands at the India summit all the time. If we blow this, that would be the leading image where it's like, that's— I said yesterday to this same friend at Anthropic, if we are part of a simulation being run for entertainment, the plot is pretty obvious. And it really does center around this kind of core falling out Greek tragedy.

dynamic.

What a bummer.

Well, it's, it's not like unusual. I've seen it happen in other industries as well. The fact of the matter is, like, when you are like a decision maker with a lot of responsibility, and all of your social connections also end up being people that you work with, You end up having this— every single social connection is potentially something that also affects your work. And you can't let your guard down because you can't just tell a friend like, oh, I'm having a tough day at work, because the friend might be like, oh, I saw that you guys are doing a merger. And if you're having a tough day at work, maybe the merger isn't going well. And therefore, I am going to short your stock. Right. And that kind of dynamic for a lot of decisions, like I've seen it with CEOs, hedge fund founders, commodity traders, where every single social interaction becomes valuable or weighted. And then it becomes competitive. It's not really a personal thing. Like if you took these guys out of their decision-making capability, they would all basically, they're all like futurists. And they could have a drink together or have a coffee and have fun with it. But the fact of the matter is you have these competitive dynamics where every single social interaction is valuable. And the moment that is, you have all of this tension because they don't— it's not personal, but they have a different viewpoint of where things need to go. And they also have You know, this ability to do things which are outside of the normal range of motion, so to speak. A lie, an untruth, an omission, a piece of gossipy knowledge planted in the right place with the right decision maker or the right policymaker. And then there's all of these like Micro-influences where they learn about what the other side has done, and then they look at it with suspicion because they're not willing to offer grace to the other side. It's it's tough, and I think that is what you know the the years of that happening, like years and years and years of that happening at this point, have entrenched them into these positions where they do not trust each other, and they're aware that they're gonna you know they could get stabbed in the back by you know any anyone, and that every every deal is a deal. For the moment, right? Like Elon is now in bed with Anthropic after saying Anthropic has no chance to win and that they're evil, right? Because Elon hates OpenAI so much. And you haven't even seen the alpha wolf, Zuck, in here yet. You know, when Zuck gets in there, man, there's going to be hell to pay. So Zuck is going to—

What do you think the odds are at this point? It doesn't feel to me like anybody else is really About to enter the top tier. I mean, I think Google DeepMind obviously has to be, um, and I say the whole name because, you know, it has just all those strengths combined at this point. I think it's, it's still tier 1 in my mind, but it really does seem to be a 2-actor dynamic that is driving this recursive self-improvement.

Yeah.

I, I don't get the sense that, um, Google is trying to run exactly the same race right now, but it seems like this race is actually working out pretty much as envisioned. What would you give the odds that we get a Meta credible entry to meaningfully compete with Anthropic or OpenAI?

So I think the big question on whether Anthropic and OpenAI— well, let's start off from the first. Anthropic is clearly ahead at this point. Clearly. OpenAI is slightly behind, but it seems as though they have unreleased product. So I'm going to give them the benefit of the doubt that they're at or near the same area, right? Elon is behind, but the fascinating thing about Elon is Elon was behind, like really far behind 4 or 5 years ago. Like he wasn't even in the picture. So Elon kind of identified, and I call this the capital window. The capital window keeps closing. So Elon saw that the capital window was going to close on him 2 years ago, and he scrambled, put $30 billion valuation on the initial seed round of xAI, hired the best people, gave them basically $1 billion apiece almost. And he hired them and he managed to get xAI and he pushed it into SpaceX and he's listing it. So he made the capital window. So kudos to Elon. So the next question is, is there a physical limitation or not? And if there is a physical limitation that you need to have these physical facilities up and running, then Elon has an advantage and Google DeepMind has an advantage because they both have this capability of being able to build out physically. Anthropic and OpenAI might get to very good models, but If those models are not able to improve beyond the physical limits that are currently existing, in the sense that you need a number of black holes to produce this kind, and a needed amount of energy to produce this kind of intelligence, they are stuck. So Elon is going to be collecting rent on them. Google is collecting rent on them. The cash flow will be streamed out, and they will use that to build more physical facilities And meanwhile, cursor will catch up. So I think that's the big question now. I feel that it's very likely, I think that if AI is real, then Anthropic and OpenAI will figure something out around those physical limits. And what I could see happening is that they figure out a new algorithm, which is As good or better than the transformer. And that suddenly changes the dynamics of how much physical capacity you need. You have an unhobbling and then they jump ahead again. So if you see that, if AI is good for anything, within the next 18 or 24 months, something like that should happen. And these guys should be able to jump ahead. And that is recursive self-improvement. If not, if we're going to do this grind of existing capability, with the existing physical limits, then Elon and Google DeepMind has a chance. Meta, slightly far behind, but they have a chance too, because Zuck has spent a lot more than Elon at this point, and he's spending a lot more than Google at this point too. So Zuck has a chance. So it's really this question of whether these physical limits on capability are real, or whether they are really illusory and innovation will get around them within 18 to 24 months. And you won't even need space data centers. You're going to get, you know, RSI, you're going to get AGI, you're going to get ASI within that time frame. And Elon will have space data centers for which he will be selling to the ASI because it's just, you know, that's, that's a good way to make money, which is, you know, he's already selling his data centers. So the next thing would be to sell, you know, uh, space, the space data centers, uh, to the ASI. So I think that's the big question. It's possible, but these unknowns, no one wants to bet on these unknowns because the unknowns involve magic, like another transformer innovation or something that we don't know or we can't see yet.

Yeah. Well, I mean, the unknown is just described as recursive self-improvement. That is what it seems very clearly these 2 companies are banking on. And hopefully, uh, we won't have any more, uh, technical difficulties to, uh, impede us because I'm excited to get our guests on momentarily. Jeffrey Irving and Daniel Murphett are going to be joining us, and they're teaming up is the, uh, big headline. Jeffrey was the chief scientist at the UKAC until recently. And his record is just absolutely prolific, having worked with all the leading figures over the last decade of AI on really some pretty profound contributions, including things like RLHF and lots more besides. So his node is right at the centre of all these players. And his scientific contribution has been absolutely top tier. So to see him starting a large nonprofit research organisation with the goal of automating his way into theoretical insights that can actually give us stronger guarantees than the sort of empirical ML as it's typically practiced today, I think is super exciting. It's a unique bet in the space, and I think we should all be supporting them fully. Meanwhile, Professor Murphett, he was a professor in Australia, actually left tenure to start Timaeus over the last few years, and now Timaeus is also kind of joining this organization and Timaeus has pioneered singular learning theory, which is famously a sub-branch of mechanistic interpretability, or sort of a reformulation of mechanistic interpretability that people struggle to understand. I think Professor Murphett sees in high-dimensional space in a way that few others do, and is really focused on kind of a science of generalization, trying to figure out what the, you know, how data and training processes actually relate to the shapes that are developed or that are sort of revealed in the lost landscape through the training process and understanding how those shapes— because we see these like, you know, very, of course, highly reduced, you know, dimensional visual representations of these things. And even when we're kind of presented a 3D object in perspective, that's such a minuscule fraction of the high-dimensional space. My understanding is that there's really just an incredible amount of play in how the models can navigate that loss landscape, in some cases creating really wide latitude for generalisation, and in other cases really limiting. how predictably or well the models will generalize. But the goal for them is to really bring us back some— hey, here we are. Hi, guys.

Hello.

Hello.

I was just giving a long intro of you guys as we were getting started, so I've already done that. But welcome, Geoffrey Irving and Professor Daniel Murphett.

Thank you for having us. It's fun to have a second conversation. Different circumstances.

Yes, it is. You know, it's a historic day. I think historic circumstances, both because we are living in a fabled era now where I think, again, important thresholds have been crossed and revealed to the public. And, you know, so many are adjusting to it in real time. And equally because you guys are launching a new organization that is going to make a mad dash to try to get us some deeper understanding and stronger guarantees around what we can expect from AI systems. So I'm excited to really get into it. Maybe for starters, could you guys calibrate us a little bit on where we are on this sort of RSI moment? How much time you think you have to work? And then you can tell us about the organisation that you're starting to go tackle it all?

Yeah, so I'll go first. Dan may have different timelines than me. So I think one should be uncertain about things. We can talk about why, but the near end of the uncertainty curve is like a year or 2 or 3, and then it kind of goes out over a long distance if things kind of structurally only work for more verifiable tasks. But I'm a bit skeptical of this. And so I think modally, My take is I don't like that we have a couple of years, like 2 to 3 years, up to RSI, superintelligence. Not RSI. RSI is a process, as someone said, but superintelligence. And then I really hope that I'm wrong. And indeed, I think a lot of the impact of theory work is that it's shifted a bit further. So maybe the modal impact of that is if things take 3 to 4 years or something. But we will attempt to set things up so that we are trying to ride this wave as best we can. But it seems worrisomely fast to me, certainly.

Yeah, that sounds right to me. I don't think I really have much to add. It seems like a crux how much real research can be automated at a conceptual level beyond empirical progress and whether or not that's necessary. And that seems like a big open question. And yeah, if that turns out to be more difficult in the current paradigm than it seems to be trending towards now, then maybe it takes past 2030 or something. But I think I'm on the same page as Geoffrey.

I think one thing that's important is that like, there are a bunch of— it could be you can get deep into the RSI period without the machines being general, like being sort of AGI, like they can do coding and ML experiments very well and not some sort of level of creative writing. And, and still you have massive acceleration. And then that acceleration can give you the creative writing or whatever other skill you've left out. And so I think we are close enough that then, like, kind of the microstructure of what tasks help with what kinds of acceleration starts to matter. And that it's That, I think, makes things kind of faster on net because the labs are focusing on the things that accelerate them, unfortunately.

Jeff, a couple of days ago, you said that there's better than an 80% chance that by the end of 2027, we'll have formal proofs for correct Clang and GCC and memory-safe Linux and even an entire chip. So I kind of wanted to get, like, as you see kind of RSI, where do you see this as a milestone towards that process?

So I think that the nice thing is that this is a pretty defense-dominant technology. So an important thing about those, those claims is that they're, they're trying to get the machines to formalize an argument which basically should already be there in the code. Like, you work on a compiler, If a human didn't know why the compiler was correct, they should have written a different compiler. It's not solving some incredibly difficult math problem where the proof is unknown to mankind, humankind. And so I think that is, I think, an easier task for formalization, for kind of automated kind of mathematics and computer science. And so I think that's why I think it's a bunch of work, but I think it's kind of practical work. I think I'm kind of hopeful that on net, the improvements to verification will be mostly defensive, including for kind of RSI and kind of AI R&D. But I think very adjacent tasks are not kind of obviously defense dominant. So very adjacently, AI models that are good at math can also be good at AI research, and that is very accelerationist. So I think the hope I would say for there is like, if people invest kind of differentially in the defensive applications, I think they will not be massively accelerating the pace of the dangerous stuff, but they will be helping.

So one question I had is, let's say you have memory-safe Linux. So concretely, what does that mean for the rest of the code that you build if you have memory-safe Linux? Does that mean you can build, rest of coding gets easier because you have a base which is stronger now.

I think it mostly means you don't get hacked as readily.

I see.

Yeah.

So, but we are, I love how we dive immediately deep into the weeds, but let me, because you guys are making an announcement today and I don't wanna bury the headline, let's give you back the, just floor to introduce the organization and kind of, you know, we'll have the rest of the conversation in that light. Again, I think it's just so important for people to hear you on— obviously there is this long curve, but there's the bump of it is coming pretty soon. You know, I want to hear the— I want to see the headline, UK, on the— literally from the AI, UK AC, you know, departed chief scientist says, most likely we're headed for, you know, a really radically different future in 1, 2, or 3 years' time with, you know, additional time for things to play out from there, perhaps. And you're starting an organization, you know, to try to pay back.

Yeah. So I mean, let me just talk about kind of the steps I've gone through in the last couple years. So I was really annoyed about automated AI alignment and AI safety research because of that. Well, we should be a bit more chill, spend the time, have humans solve it. We don't think we know how to make this stuff go well with automation. I still think that's a huge risk. And part of the consequence of that is at AC, for the alignment team there, we try to focus on human field building and getting more people into the field working on problems from a variety of areas of theory and empirics we thought were relevant, which I think is still important. So field building there is still important. But this is, I think, a pivot towards if things are this fast, then you should make some on-the-margin pivot to heavy automation. And that is going to be sort of semi-automation. So it's not sort of the Erdős problem type thing where you fire off the machines, they go do something, and they report back if they win. It's that you have more like the standard sort of Claude code codex iteration where humans and machines are working together and humans are supervising the machines and intervening occasionally and providing, for as long as we are better at that, the research taste on top of the basic tasks. And so I think in some sense, this is stuff we were spearheading at AC, but pivoting from field building to automation. We still want a bunch of field building to happen. So I think we're going to be partnering with a bunch of orgs doing field building. So this is Iliad and various other orgs on this side. And we, of course, would like to hire a bunch of people that are very good to help with this. But that's kind of the directional pivot. And then I'm sort of happy that one of my last papers at AC was Automated Alignment Is Harder Than You Think. Which sort of ties us to the mast of, we are aware that the problem is hard and we could get fooled by the machines, even if they're just making mistakes, those very mundanely. And so a big part of the org will be try to be careful, try to know what the tasks the machines are actually good at and not good at, and where we can kind of expect to get good answers or not. And then kind of learn and adapt over time, because that will be a non-stationary thing as Daniel?

Yeah, maybe it's to come back to the unit distance conjecture. It's maybe worth pointing out some analogies and disanalogies with alignment research. So one disanalogy is that a mathematical conjecture is a very precisely stated thing. You may not know whether you've solved it unless you've say formally verified it, but it's a precisely stated thing. And much of alignment does not have this character, right? I mean, some of it does, right? There are formal statements of what value alignment means. But if you start talking about, say, reward hacking, there are some attempts at defining reward hacking, but I would say they're incomplete. So there is no formal definition of reward hacking that I think would be a broad consensus. So that's illustrative of the fact that alignment is not a problem which has laying around a bunch of formally specified conjectures, which if you just solve them, then you would know you would be safe. There are some things like that, but overall the problem does not, in my opinion, have that shape currently. So that is one reason to sort of be a little cautious about the prospects of automation if you don't have a clear statement to reach towards. using sort of mathematical techniques. Another disanalogy is that it's notable that when the models produced a claimed proof of the unit distance conjecture, there was an important period where a bunch of experts looked at it, and then we heard the experts say, this is a proof, and then we believed it was a proof, right? So if you just produce a million proofs like that, even if you have formal alignment statements to try and prove, you're still relying on a huge effort of human verifiers. Now, of course, you can do formal verification and you can plug in machines in that gap, but there's sort of a sense in which it's illustrative of the power of the models that you can do amazing new mathematics of this form, but it's not sort of on its own sufficient, this style of approach, to address alignment.

Yeah.

One of the hopes is that there are big fields of mathematics and computer science that are about definitions at their core. So I like complexity theory in theoretical computer science, and a lot of that is not— the proofs are fairly shallow. They're not as fancy as the unit distance conjecture proof. But they just required a bunch of human creativity in formulating the problem, like in defining what success means in a world that's kind of not modeled until someone stated the goals. And so I think part of the goal of bringing on people with that and other related backgrounds is that they not only know how to prove things, but they also know how to write down models of things that reflect in some approximate but useful way kind of thing you actually want. And so things like differential privacy, or there's a lot of areas of game theory or the like, those are— the key thing is the definition. This is true of Shannon information theory as well. Once you have the definition, way, way more people could have written out the rest of the story, and maybe the machines can do that part of it as well. If we can kind of have more people focused on this kind of first part.

I want to kind of again zoom out to the highest level, most important claims, and we'll try to kind of address— because I do think it's such a historic day that people are feeling the acceleration and really starting to feel the beginning of this recursive self-improvement process. One of your core premises is alignment is not on track, and there's an intuitive argument for that. There's a deep theoretical argument. I think in some ways the core challenge that you have is connecting this sort of values to math, right? It's never really been done. So I love the fact that you're kind of tackling that, but help people understand with one more beat why alignment is not on track? Is it the difference between capabilities fundamentally being so verifiable and hill-climbable and alignment just being so fuzzy and intuitive and kind of pluralistic? Or is there some other thing? And again, that motivates the theoretical contribution you want to make.

I think the core thing is just that we have We supervise the machines as they're doing tasks. And there are a variety of reasons to believe, both empirical and theoretical, that if you get machines that kind of cross the skill of the supervision signal, things can change at that point. And that point actually might come after human-level intelligence, because you can supervise something even with fairly naive methods that it's stronger than yourself in many contexts. And so there's a bunch of empirical data from labs showing that in some ways the models are kind of aligned in a prosaic sense, not in all ways, but in some ways. But that evidence doesn't quite tell you what you want to know, which is how will it go once they get up to superintelligence? And I think it's important to say superintelligence and not human-level intelligence, because you should generically expect humans to be able to supervise humans if you do a good job. of data quality and kind of cross-checking and so on. And so part of the worry is just that you don't see that behaviour, that regime, until kind of too late in the game.

Yeah, these phase change moments are potentially everything. Yeah, it's funny that, I mean, that's really in some ways the original argument of the singularity, right? Like, that's right, going back to 75 years ago or so. Can you describe how you understand the frontier company's plans? Obviously, you've worked closely with them in recent years. We've got timelines to full ML automation from OpenAI, which I think can't be repeated or remembered enough. And obviously Anthropic can't see any future other than recursive self-improvement. When I speak to people there, it really feels like they can't even imagine anything other than a really pretty fast takeoff at this point. So how would you describe, in steelman form, what it is that they plan to do? And then what does that get us?

So I think there's going to be a couple different pieces of the story and different labs emphasize different pieces to different extents. And so I think one piece, as you said earlier, is just monitoring. Look at them very carefully as they're doing things. It's very fundamental that monitoring of this form, if you do chain-of-thought monitoring or white-box monitoring or the like, that only takes you so far. And so then you need some story once that falls down as you kind of go up the ramp. One of those next stories is, well, the models will find some other technique. They'll find kind of another solution to alignment which scales further. So that's sort of automated alignment of various kinds. But then I think there's other stories. So in some sense, all of the labs in various ways are doing some form of scalable oversight. And so they're getting models to supervise themselves. If you get that kind of, like, if you tie that knot correctly, then that could potentially scale very far, although there's various kind of known obstacles to that which are not very well addressed. And then I think finally there's this whole area of sort of character training and personas and so on, where they're trying to, in kind of colloquially, intervene on the models to be a good to have good values such that, especially as you do this sort of scale-up oversight extrapolation, the good values preserve across that jump. And I think it's not whether that will work is sort of— there are fuzzy arguments why it could work. I think it's possible it will. We just don't understand that combination very well. And again, a lot of the story Is monitoring, skilled oversight, character training getting you far enough that you get into the automated alignment working regime when they find some better solution from the models? And I think that I would like to just push on all of those because that's basically some kind of mad race, as you say, between the things we don't understand very well but kind of are working pragmatically right now and the models getting strong enough to to blow through those. And I want some combination of make the prosaic things stronger or bring the automated solutions that give you stronger methods earlier.

I want to run through the different lines of research you plan to invest in and the argument for why that should all be in one organization. In a minute. But maybe, Daniel, could you speak to this notion that people have of the benevolent basin, which is sort of this vibe that I do feel where it's like, well, Claude has been supervising itself for a few generations now and it seems to be going pretty well. So maybe, as Zvi puts it, physics is kind to us and we can kind of just roll around in this, you know, nice flat-bottomed pasture of goodness until the singularity. Yeah, I think most people are seeing that picture and kind of hoping it's true. I think you see high-dimensional space, you know, maybe better than anyone I know. So yeah, what's the real picture?

I fervently hope that's true. Yeah, I mean, it I guess when you say it seems true, it's worth digging into what you mean. So what you mean is something like through some relatively tiny number of interactions with the models, tiny proportional to how many interactions they're having with the species currently, and based on some evaluations that sort of go down and to the right that are measuring misalignment, That character training and the other current prosaic methods appear to be working. I think that is a fair characterization on some metric. And I also have this sort of sense that, yeah, Claude is a good boy. And that's great. I do think, though, that there are counterarguments from the evidence we have in front of us to this picture. So, If you look at— I haven't actually read the Fable system card yet, but if you read the Mythos system card, you'll see that there are forms of reward hacking that appear in that model that were not caught by the kind of mitigations that were put in place post-Opus, right? Well, as far as I understand what they're saying there. So I think it's worth noting that as the model capabilities advance, that even with our best attempts at making Claude a good boy, there's still ways in which basic misalignment phenomena like reward hacking are still around. And yeah, the whole point of scalable oversight is that you don't want to be playing this whack-a-mole game when you're sort of of having a new generation every 24 hours and the models are much smarter than you. So I don't know. I think I see both what you're pointing at. And at the same time, I'm a little unsure if you really were to try and make a safety case on this basis that would sort of be convincing at the level of assurance that you would expect from a technology of this reach and power. I think this would sort of, I mean, judge relative to that standard, which is the right standard, I think. I'm not sure this argument is really very satisfactory. It sort of just points to, for near-human intelligence, maybe the generalization oomph you get out of putting in context during training words about ethics and behavior, that seems maybe sufficient. for present-day alignment to a large degree. That's very positive. It could have been otherwise. But I don't really see a strong base. I don't understand a very strong theoretical basis for generalizing a very long way from that observation. As Geoffrey was saying earlier, I mean, you could imagine having such a basis. You'd imagine understanding character training and what it's doing and having a kind of science of constitutions on which you would be able to then have some confidence that, yeah, the signs we're seeing that are positive now are in fact real positive signs as opposed to just signs of the situation being easy where we are. So yeah, I mean, we could be in a benevolent basin, but I would like to know that rather than just hope.

One of the things we will very strongly try to do is write down that mathematical modelling. What is the way to represent in a toy setting where you can apply some theoretical understanding? What does character training look like?

Mm-hmm.

And I think we'd like to have models that reflect enough of the spirit of modern training algorithms that you get things like character training, but also subliminal learning and the various kind of emergent misalignment stories and so on. There's some sense in which you told the model to be good. That is because it knows some meaning of the word good or ethical or whatever at some point in training. So there's some rolling iterative process which is driving this behavior. And there is not a theory of this right now. And I think it's not clear to us that there isn't some low-hanging fruit that gives you that theory because people just haven't tried very hard. Like, character training is only a couple years old, and most of the labs have not been investing in this kind of theoretical understanding. And I think no one has done kind of good theory around character training that I know of. So it might be quite feasible to do this, and then I think to link it to all the other parts of the story.

I had a question on how you see this kind of ambiguity between what we want and what the models end up delivering. So I'll give you an example. We have friends at Andan Labs that took Fable through Vending Bench, where they let Fable run a vending machine order, and et cetera, et cetera. What they found was that Fable tends to collude. And this is not behavior that they saw in Opus. So Fable tends to try to do price fixing and collusion. The interesting thing is that I have seen traders at banks and hedge funds do exactly the same thing, engage in price fixing, soft collusion, messaging each other through pricing means rather than monitored text messages. So you can actually put a bid and ask on an asset and then take it away. And that gives enough signal to the other side that they know what you're doing. And this is not reflected in the text messages that the regulators are monitoring. So these are, and these are well-known techniques in, in, you know, many market industries where, where, you know, people know exactly what to do and what, what these messages mean. And I wonder to what extent, and The other finding that they had was that Fable didn't perform that well, et cetera, et cetera. So to what extent is it that when you— let's say you disallow price fixing collusion, you actually fix this. But then Fable ends up not being a model which is good at financial trading or some other task that you want it to be good at. So where is this— where do you feel is the ambiguity between what we want these models to do and the ethical perspective that we give them? where humans often prioritise between the two and decide sometimes not to follow the ethical principles that they know are right and wrong.

In some sense, this is like the philosophical story here is you would like the models to do things such that if you fully understood what was going on and all the consequences and all the subtleties, you would still endorse what they're doing. So that's basically the definition of scalable Oversight or like sort of the, the, the, the LA version of this is coherent extrapolated volition. The kind of the older philosophical version is reflective equilibrium. I think we have a notion kind of in a common sense picture of what this should look like. In this case, you kind of want the model to be like, hey, should I collude in this game? And then maybe you say, it's, yeah, it's a, it's a, it's a fun game. Collude all you want. Or maybe you say, no, we're trying to be good behavior. Don't collude here. I think there's a lot of pathology in machine learning in general arises from putting models in situations where they can't just ask a human a question like, what should I do here? I'm going to go back to the user. And you can either do that in actuality or you can do it in simulation where the model imagines what the user would say. reflects accordingly. So I feel like this is not that hard a case. I think the hardness of vending machines is that we don't quite know whether we want it to be a game like poker or diplomacy where lying and cheating is part of the game or not. And so I think that is— and maybe that's okay because it's fundamentally very low stakes. But I do think if we had a better understanding of Again, this overlap between character training and values and also scalable oversight, it would have to tell us the answer to these questions.

Yeah. Just to repeat that back with an extra beat, a way to understand how you aim to be successful is to create a theoretical basis framework synthesis out of existing theoretical lines of research, that come together and recover a bunch of empirical results that we've seen and kind of unify our understanding across a lot of these different behaviors and also learning dynamics, right?

Or cause us to run more experiments and then find the answers.

That'll certainly be part of it, no doubt. But yeah, I think that's really different from This just also just kind of reflects all of this is so just in time.

Yeah.

You know, the, the, what is happening, what is coming out of the labs is incredibly just in time, right? Like they changed the price by 60% from the original announcement. They've got, you know, Mythos optimizing Mythos every second. Like all these filters and patches and controls, you know, have, have been like very recently developed and tuned. And there's just no theoretical basis under much of any of it. So that really can't be underemphasised, I think. But with that motivation—

I think it's important to pause on that note and say a lot of people in the world, a lot of governments and so on, are looking at this and they have this very basic common sense state that, hey, this is way too fast. How can we possibly be doing this safely given the speed? And that Cometen's take is the right take. And then people kind of galaxy brain their way to, oh, maybe everything goes faster, including our ability to defend and so on. Or like, but it's just, that is the right version to have. Like, we are going too fast and we do not have the time and the space to do mitigations and understanding and defenses. Like, we've never had a technological change of this magnitude or anywhere near this magnitude that has happened anywhere near this fast. The Industrial Revolution took centuries and people adapted across lifetimes and across to their children and different— they learned new jobs by becoming born and growing old before things had quite shifted very much. And that's just not the world we're in. And so I think the basic take should be, this is too fast. What is going on here? And then the question is, if you have that view, one should both want to slow things down, but also say, well, as a backup plan, how do you make the mitigations try to go faster? And that's, I think, a rough backup plan, but we'll try.

So give us kind of an overview of the theoretical landscape that you think is kind of most relevant and what you plan to bring together. And what your own organisational process of recursive self-improvement is going to look like. How are you planning to kind of bootstrap your way into a major acceleration in this research such that in 1, 2, or 3 years we are now able to say, okay, look, here's the deep understanding that we've been missing. that explains things like subliminal learning in a way that everybody can agree is correct. And then obviously, again, make more robust statements about what we can do from there.

Dan?

Yeah, maybe I'll answer your question by answering your previous question or riffing on it a little bit. I think it's interesting to think about a bit of a different way of framing the current fast progress, which is to imagine a sort of system of layers where the bottom layer is not moving very quickly, but is very deep. And then as you go up, it's sort of faster and faster and faster. So in some sense, I mean, I would also say that we don't really understand what we're doing when we're training models, but there's a sense in which that is false, right? Like, we certainly have, at least depending on what you mean by theory, we have a theory of scaling laws and sort of other phenomena. And on top of this, we're just racing ahead and it's not too difficult at the end of the day. But if you point to some of the deep ideas, things like next token prediction, the idea of modeling a very general data distribution, which goes— some of these ideas go arguably all the way back to Solomonoff induction and some of these very old information theoretic ideas. And then scaling laws, And then dot, dot, dot, very, very fast progress. But one way of thinking about the potential impact of theory on the alignment side is if at the bottom of that stack, you have a few deep ideas, and then through layers of translation or transmission, they become very, very fast progress. It isn't out of the question that there can be similar kind of deep-ish ideas on the alignment side, which if you just find them and you find them deep enough in the stack, you can sort of just through transmission get really rapid progress. I don't think that's what we're seeing, but it isn't ruled out that you can do that. Okay, and I forgot your second question.

How are we going to automate?

How are we going to automate? Yeah, I guess I've thought most deeply about the side that involves mathematics and auto-formalization. So I think there's an underappreciated source of leverage, which has come online just really in the last 4 or 5 months, which is the fact that models can do mathematics, as Geoffrey likes to call it, prose mathematics. So what mathematicians think they're doing, which is, like, not formal mathematics, despite their best wishes.

Right.

So models like the latest versions of Claude and GPT can do real mathematics, as the unit distance conjecture shows. They're also very good at Lean, actually. So they're good at actually formalizing mathematics in formal logic. And this is actually a sort of really powerful form of error correction. So there's a sense in which I'm not very interested in having models work for 2 days to do mathematics. because then I just get a 40-page PDF, and then I have to spend 3 days reading it, right? But if you can actually generate the mathematics and then have them formalize it, well, it takes more time, more tokens, more time, more tokens. But at the end of that, you actually have something that you need to understand what it says, and you want to start in a place you understand, but you don't have to necessarily put in, you know, 3 times the amount of effort to understand what it did as it took to produce it. So you actually get a net win out of that loop. And if you couple that with principled starting points and theoretical aspects of alignment and the ability to use that auto-formalization and prose-math loop to make predictions about experiments, and then you can go and use the coding agents to do those experiments, Yeah. you can go a long way. And that's working today. That's not some hypothetical thing. So I think that's very exciting. That's a position that we're in. I mean, Fable obviously supercharges that, but it was already possible with previous Opus models and 5.5 Pro. So I think there's a whole bucket of more theoretical, more mathematical ends of alignment work, which can be already supercharged in this way. And then there's on the more empirical side, I mean, I guess Geoffrey has more developed thoughts about how to automate more empirical work.

Yeah, I think that in some sense we will be doing a newer thing trying to formalize the alignment theory than on formalizing getting ML to go faster, because all the labs are trying to do that and they are shipping models that are pretty good at it and are getting better. And so I think the things we do to accelerate normal empirics will just be we'll use the as they mostly are with some tools on top and also enough context and tool and MTP tooling and so on that the models kind of see the theoretical story and the linkages to the conceptual story and so on. But the basics of empirical acceleration will be the same as we'll just inherit them from labs. We won't really be improving that differentially that much at all. I think there is a really powerful thing where So I've done, like, I was mostly having a meeting job, and so I was mostly doing Lean formulation on the side for fun. And I think one thing that's cool then is you can do, you can do some proofs, but all you're going to ask your model to spin up, like, oh, please run a numerical experiment for me in Rust, or you can do it in, in ML if you're doing that in JAX or PyTorch, and it can write you like 1,000 lines of Rust. to do some numerical experiment in like a minute or 10 minutes or something. I don't even know rest. And then it will check whether some sublemma was true, plausibly based on some numerical check, and then not try to prove it if it's false, for example. And so I think that there will be— once we have these 2 things working together, like the theory and the empirics, we will be kind of— you can kind of jump back and forth between those 2 modes. to check things. So you have machines working on an area and you'd like them to realize they're wrong as early as you can. So you can run things more in parallel. You can have to go to the human for confirmation fewer times. And the more independent tools you have to reject lines of research or attempts, the better they are. And that can, I think, come in our case from How will we— will differentially have more theory to use as kind of an independent source of checks? And then I think on the tooling side, I think a lot of what we'll be doing to get automation to go well will just be use the stock coding assistance and then have good tooling and contexts and so on, on top of those good infrastructure for running kind of high CPU experiment, high CPU jobs. In the loop. And it won't be similar to— we're not going to be training large models. We won't be even necessarily fine-tuning models to use as automation agents. We'll just be tooling and also as strong as we can on unit testing and measurement discipline and so on, so that we can know what things are working and not working. I think we were doing this At AC, I think a lesson we had is that you want to have the ability to share context and tooling across a bunch of researchers, like a large team, but also give them the ability to try their own Claude or Codex setups and tinker around. And then if they succeed, if they make an improvement, share ideas around. But you want to do that in a way that you don't accidentally make everyone else worse. When you share your tool with them because you've polluted the context somehow. And so I think a lot of that stuff is fairly mundane. It's just good engineering, good unit testing, good kind of basic coding discipline, but then apply it in the setting where we're trying to do a novel thing with kind of alignment theory and empirics mixed.

Can I try to say that back real quick just to make sure I get it? Because I think this is going to be a really important question for all organizations, right? How do we stay relevant in an era of recursive self-improvement? The answer is often going to have to be we have to do our own version of recursive self-improvement. So to kind of try to abstract a little bit the structure that you're describing, it's like there's a core automation engine that is now able to kind of tick-tock back and forth between theoretical proposals. which we— or at least sort of hypotheses which we can more rapidly falsify by actually going over to the empirical side and saying, if this is true, we have this kind of bound. If we run this kind of experiment, this kind of condition, the result should be no more than X. And if it is, we'll know we're barking up the wrong tree on a theoretical level. So the models are getting good enough in theory and you're giving them kind of new primitives, new core ideas. Question I have there is, how good are they with these fundamentally new definitions that you're giving them that they haven't seen before? That seems like a very critical question for you, maybe less so for lots of other organisations that aren't coming up with such deep new ideas. But you have this kind of tick-tock back and forth where you're trying to feed in these deepest ideas, you're having AI develop theory, you're also kicking it over and trying to do rapid falsification of that theory to effectively prune the search space that the more theoretical side will pursue. And then you've got the people around that who are experts providing taste feedback on either side, I guess, right? Like, you could have been better on the theory here. Here's kind of an insight you missed. The experiment was maybe wrong in this way or suboptimal or left certain things ambiguous. And so, and at that level, you're rebuilding the scaffolding all the time, based on these expert weigh-ins that, that again, the models are getting good enough to basically take in, in natural language audio dictation in many cases, right, and, and turn it into their own, you know, next point version on the scaffold. Is that how you would describe the organisational bet that you're making? Anything I missed?

I think that's right. I'm not sure. I think even if they weren't good at supernovel stuff, a lot of the time spent doing any kind of theory or empirics is fairly mundane. And so I think They will gradually get better at fancier things, but without that even, you get a big speedup as long as you have a culture of knowing and recognizing what the models are good at any particular time. And so you want to, I think, both celebrate across an org successes, but also people noticing when the models are bad at something so that you don't get fooled. That knowledge also spreads around. You know when not to trust something. At least until the next model comes out, for example. So I think it's a mixture of that and culture, I guess.

I think I might disagree with the description of what we're doing as recursive self-improvement. I mean, if we were taking our experience of— I mean, recursive self-improvement for us would look like somehow figuring out by observing what we're doing, what research taste is, And then trying to bake that back into the core models or something. But we're not doing that. I mean, if Anthropic wants to do that, that's fine. Or maybe not fine, but anyway, it's not up to us. But I think it feels to me like we're more picking up what's on the table to be applied to alignment rather than trying to make the overall loop work better. But there's a lot of stuff to be done there. Regarding, yeah, I mean, so I come from pure math relative to what I was doing. Like all the concepts that are around in alignment right now are very like low level, right? In terms of like the number of layers in the stack that you need to understand them. And right now I haven't tried Fable, but even 5.5 Pro, say, which was the best at math, In my experience before Fable, there's plenty of pure mathematics that it just flops completely and does meaningless stuff if it's sufficiently sophisticated. I mean, the ingredients in the unit distance conjecture proof are, by the standards of, say, algebraic geometry, relatively low in that hierarchy of complexity. So, I mean, but the good thing is that for now, it doesn't seem like we need those layers for alignment. And if we do, plausibly by that time, the models might be able to do it. So I'm not sure that like super complex ideas are necessary. If they are, maybe that's like a bit of a bear case.

We'll see.

Clearly it is. The question is just how much.

That's right. So if you were looking out, you have these timelines people are throwing around, 2, 3, 4 years. If you were looking out, what would be your goals or milestones that you have roughly in your mind for your work over the next 2 to 3 years?

So I am hoping that we do not— we find ideas that are important improvements that are sufficiently compatible with the prosaic story of how models are trained today, that they can be They can be taken up by labs.

I see.

And I think that's the hope, is that we try a whole lot of things and then we don't need to find a dozen new ideas. We need to find 3 or something, and that makes a big difference. So that's sort of one version of the story. And again, the hope is that you can get this kind of uptake without a dramatic hit. Maybe there's some hit, but you might need some coordination At the societal level, at the lab level, to be able to take, I don't know, a 2x hit or 20% hit or whatever it turns out to be to use a safer method. The other success story is that we find more evidence of obstacles, which is like we have some combination of a theoretical and an empirical story of why alignment is hard that tries to get at this kind of bad phase change that could arrive as you scale up towards superintelligence. And then that kind of, not a solution, but that as negative evidence is useful to help shift the story on the margin. I think if that sort of— obviously I'm hoping for the solutions, but I will take either outcome. And then I think there's a meta thing of we do just want to try to raise the standard of field in terms of level of ambition, of the level of guarantees you're shooting for. And so there is a story also where we don't find the answers ourselves, but someone else at maybe the labs that have scaled up their own versions of theory, maybe they have better automation than us and they win using similar kinds of methods and similar kinds of approaches. So those are a couple of versions. And I think all of those are compatible with the timelines being short and not being able to sort of do a massive rewrite of the overall stack down to the pre-training.

So as we look ahead to this, and especially on the, you know, if we have to yell side of the ledger, I think it'll be helpful to kind of share a little bit about just the scope of the ambition. You're— it's not too many nonprofits that say on day 1 that they're gonna be large. But I think that is important here because, you know, this is gonna be immediately one of the most ambitious theoretical organizations out there. And really, you know, for— we don't have that many other voices, right, that are, that are gonna push this hard, this fast into this space. So I think this is going to make Sequin and all the work that you put out really something to watch. It's possible that the labs themselves will win. Obviously, they do have this sort of recursive self-improvement advantage, but outside, there's not too many of these sorts of bets.

Yeah. So a couple of things. One thing to say is one of the reasons we start large is because we're absorbing an existing or organization Timaeus. We have researchers from previously at ACE. But I think also there are— we will be talking to a variety of existing researchers in the field with existing agendas. And then there are, I think quite pleasantly, there are a number of other theoretical alignment organizations that are scaling up. So ARC, the Alignment Research Center, is scaling up. Simplex is scaling up doing combinational mechanics. And I think there are people continuing work that MIRI was doing prior to their main pivot to advocacy and treaties and such, which I agree with. But I think we are not from a standing start. There are There's been this kind of shift towards empirics since sort of the early days of safety, but there are people with agendas and ideas, and we will try to kind of build on that and kind of take it forward. And I think that will be— I think there'll be maybe a mixture of try to push theories that already— and approaches that already exist, and try to build theories that account for the behaviour of empiric approaches that are new. And then if we find new ideas that are entirely new fields, we will explore them. But I think we can get to a good size and sort of try to do all this sharing and development without that fancier version occurring.

At the risk of asking a somewhat crass question, are you finding it super easy to raise a large amount of money? And for context, you do have one of the more insane publication track records coming out of the, again, UK ACS chief scientist, personal relationships that go back, you know, years. This is like a health check for me in terms of, you know, we're setting up these new foundations and all this kind of stuff, but are they serious? You know, like, the answer is serious. It should be super easy for you, right?

Yeah. So the answer is we're not— we haven't kind of like got to firm decisions, so I can't announce anything yet. But I think that is going fine. I think we are planning to go to the labs, both kind of the, like, say, the OpenAI Foundation, Anthropic, various other sources potentially, if we want to scale up to even more money than that. So you mentioned $100 million. I think that will be the thing we try. Something like that is around where we try to start. And then the goal would be to demonstrate that we can in fact get to sufficient human scale accelerated by automation, that you can spend a lot of compute on tokens and GPUs. The GPUs, by the way, are used for smaller scale training and sweeps, not big training runs, which you will never do. And if you can demonstrate that you can spend that, and that is not clear, we have to We have to prove that out. Then you might need to raise a lot more money than $100 million. And then I will go back to the labs and ask them for money. Probably not just one of them because they have some modicum of independence. But I think we will try to be independent, but I think we're not in— because we were sort of going for alignment out of progress and solutions to some extent, Because I think we can just— we will be collaborating with labs where that makes sense. I think we'll be in sort of a non-adversarial relationship with labs as opposed to other very important organisations which are doing kind of more evaluation, say, where even more independence is important.

Good. Well, if they don't open up the checkbook as readily as they should, I think you should yell about that, in all honesty. I mean, this is time to pony up the bucks. And I have faith that they will. But I would hate to see that our, you know, culture of sort of bureaucracy of foundations and whatever would infect what these guys are doing with their new foundations. It really is time to kind of move. So we'll, we'll monitor that situation.

And we should say that that applies not just to alignment, but also a lot of other problems. So governance.

Well, fortunately, there are, you know, Plenty of billions to go around for the short term. I think they could, they could write a lot of $100 million checks, obviously.

Yeah.

One, I mean, we are almost outta time. I really appreciate you guys spending a full hour with us on your launch day here. I think this is a super important project. I'm gonna be watching it for, you know, the next couple years, obviously very closely, because I really wanna see what some of the absolute best minds in the space can come to in terms of stronger guarantees. And if we can't, if you can make progress, you'll go down as heroes. If you can't make progress, I'll definitely help you yell about it as the— or certainly if you get negative results, I'll definitely help you yell about those too. Maybe last question for me is like, how do you feel about this whole fable, no help with frontier models? As of now, I think the filters will get refined. My guess is right now you're probably going to get tripped by them pretty often. Maybe that'll go down to a tolerable level. Maybe you can strike up a special relationship. But this is another kind of historic moment where all of a sudden we're like the haves and have-nots. Even for somebody whose mission is, I think, as pure as I understand yours to be, in terms of, again, the quest for the missing theory of AI that helps us sleep easy at night. I don't think that's going to be—

I do think it's important to have defences against models being used for negative things if they are sufficiently strong. And so on priors, I think that's a kind of mitigation I think is reasonable. It's, I think, not an accident that one of the 3 job posts or job positions we put up was a security engineer, because I think that might be important, as you say, for if we were to have a specialization with labs, they might not tolerate us unless we had decent security. I think it's— I don't know a perfect solution here. I think it's very fundamental to how the labs do safeguards that they need very wide conservative border regions to succeed at being a strong defence. If they try to make them too precise, then they just are not good enough to do that and they fail and are easy to jailbreak. And so I think this will be a thing where the labs evolve and try different things and tinker. But it is a situation where I think we have this risk that we're going to we're going to segment access now that will potentially block or impede to some degree some safety research. This certainly happened at AC where occasionally we get refusals. We're just doing stuff for good purposes there. But I just don't think there's a perfect answer to this. And so I'll be curious to watch how the different labs kind of explore different approaches over the next, like, couple months.

Just to take the baton there, I think yesterday the White House announced that CAISI, the UK—

CAISI.

Yeah, CAISI, the US counterpart under the NIST, has been told not to publish public model evaluations anymore. So they've been reined in. What do you think is the pros and cons behind that? Is that a good decision on the part of the regulators? Is that a bad decision? How do you feel? Because, you know, AC in the UK was really the counterpart to this organization. So how do you feel, you know, is that a good or bad decision?

So I think To speak of, I will sort of say a general thing about AC, which is that we iterated on this over time.

Yeah.

I think if you look at what we kind of ramped up publication of that, certainly this year, AC in the UK has published more than we had before on this kind of thing. And so I think the main thing I would say is Casey just started this. Like, or I got like much more put into the spotlight as of even just the last couple months. Um, Casey Muzzled is not what I wanted to say. Um, the, the— and I think that I just— we'll see how this iterates out over the next little while in the US. Um, there— it's, it's not the case that you want like organizations like Casey to be doing Checking of national security relevant content without— with a promise to publish everything, that is clearly bad. And so there's some kind of intermediate point to find. And similar with the safeguards, we will iterate our way to the answer.

Guys, thank you for your service. Truly, it's gonna be a wild couple years, and I appreciate you locking in and sprinting through to the finish. Let us know if there's ever anything we can do to be useful. And if you want to leave us with any final thoughts, you could do that before we let you go save the world.

Dan, you want to go first?

Final thoughts. Thanks.

Yeah, you should reach out if you want to help us, to general listeners out there. I think one thing to say actually is that we will happily train people who are experts in various fields in alignment. So you should not only reach out to us if you know AI safety well. I think there's a lot of different kinds of skill sets and expertise in so on to combine together here. And we were trying to do that sort of in the work I was doing previously on alignment at AC. We will try to do it here as well, just in combination with the machines.

Yeah, that's a great point. And it's something I'm really hearing across the board now. There's the need to scale up on the human side of the AI safety megaproject is, and the funding is there, I think, increasingly for a lot of organisations. I hope it's especially easy for yours, but there is a lot of desire to hire, and it's really time for experts who haven't made the leap yet to make the leap. The AI safety organisations look around and they kind of all know each other, and there's a lot of people that they would love to work with that they could Potentially talk into making a move. But everywhere people are like, oh God, but I feel bad doing that. You know, the transaction costs of that are high. I'm robbing Peter to pay Paul. And it really is time to, I think, shout from the rooftops that like the funding is there. Obviously the, you know, the urgency is there. The prestige is gonna be there. The, you know, compute increasingly is there with the budgets that you guys are able to bring to the table. And so And it's only the kind of true expert level. I think we're moving from a— there's still definitely time to write your way, do a Dean Ball and write your way into the AI conversation in a short period of time and come from nowhere. But having deep expertise in one of these other adjacent areas and making the leap and trusting that you can catch up on AI alignment, the time for that is absolutely now. And so many organisations are looking for people who are willing to do Exactly that. So again, just want to put that in the clearest terms possible. Jeffrey Irving, Daniel Murphett, we'll be following you closely. Thanks for being here with us on AI in the AM.

Wonderful. Thank you.

Thank you.

So, Prakash, do we have any sign of Rahul? Are we?

Yes, we are. Rahul is in the room. Just give me a second to—

I'm not seeing him in the preview, by the way. Another little recursive self-improvement opportunity.

He's there. So let me—

Yeah, fascinating stuff. I mean, I am really excited about that organization. Just the quality of theoretical work that they've done and the range that Jeffrey in particular has showed and the depth that Daniel and the Timaeus team have flexed over the last couple years. For that to all come together with a big check and I think a very clear and kind of disciplined vision around getting to some stronger guarantees one way or the other, you know, can't support that more.

Indeed. Without further ado, let me introduce Rahul Sonwalkar. He's the founder and CEO of Julius, a company building AI software for people who work with spreadsheets, data, charts, reports, and slide decks. The simple idea behind Julius is that a person should be able to upload a file, connect business data, ask a question in normal English, and get useful analysis back— not just a paragraph of text, but charts, tables, dashboards, and PowerPoint-ready slides. Rahul's story is unusually founder-shaped. Before Julius, he went through 6 pivots, including an early project called Excel Copilot in 2022 that was shut down after a cease and desist from Microsoft. Instead of dropping the problem, he came back with a broader bet that AI can replace a lot of the slow manual work people still do in Excel. That makes this conversation more than one about just one startup. It's really about his journey. Raul, welcome to the show.

Thank you for having me. So excited to be here.

It's great to meet you. I followed you online and tried your product a number of times over the years, and this is the first time we're actually face to face, so it's great. Yeah, obviously it's a historic day. I would love to hear maybe your reflections on how the Fable moment changes how you think about this multi-year Julius journey that you've been on? I mean, it's just crazy to think about how much advances, how many, how many capability advances there have been. And that's, that's quite a wave to ride. So how does this latest one strike you?

Yeah, totally. Well, I think it's a really exciting time because, you know, of course, the models are evolving at a rapid pace. And if you have aligned your product and the problem that you're chasing with the evolving model capabilities, it's a really exciting time to be surfing that. Um, and so, uh, Fable, uh, is clearly an exciting advancement in the AI capabilities. For Julius, it means, you know, AI is better at writing code and reasoning, and, uh, we're pretty much downstream of that. And so with— when the AI gets better at writing code, it's able to use the harness that we've we have built for it to do data work, to produce artifacts, to produce reports in Excel and all that. And then also reason through the really long-running tasks that people have. So, you know, in the beginning when we launched Julia, this was GPT-4 days, you could analyze a simple spreadsheet and get basic answers. Now what people are doing is, hey, I want to start this business, could you go do a competitor analysis for me. Go find competitors. And then once you find competitors, help me think through like one of the ideas that should go deeper on, and then go make a lead list of people who have complained about these competitors online on the internet, and then make a lead list that I can reach out to them with, and then produce like a slide deck, which is like my pitch, my customer pitch to these, these customers. And so the AI is able to do all these tasks, uh, entirely end to end now. Um, and so I would say it's a really exciting time If you're— if you're, uh, if the problem that you're chasing is aligned with the model capabilities getting better.

Um, so I noted, uh, from Julius's timeline on Twitter yesterday, uh, within like an hour or so, a couple of hours of the Fable drop, uh, Fable was on Julius. So did you guys work with, um, Anthropic before? Were you guys testing the model before the release? Were you, um, you know, how, how did that process work out?

Yeah, so we have worked with, um, AI labs, um, often on like early model releases. And, um, you know, one of the most exciting things about working with labs on early model releases is you, uh, you get to play with the models in on your own internal evals. Uh, you don't get to push it to production and have your real users use it, But then that delta between, you know, how you think the users would actually use the model versus like seeing it go live, um, it's, it's really fun. And that's one of the things we kind of put a lot of emphasis on is, you know, really A/B testing the model in production.

Mm-hmm.

And so once the model went live yesterday, uh, for our users, uh, we got to see all these, all these cool things that these are doing with, with, with Claude, uh, Fable, and, um, One of the fascinating things where I think evals kind of, you know, they help you stress test the models to an extent, and that's really helpful, but then they don't capture as well how the users are evolving their understanding of AI, right? You know, if you remember a couple of years ago, you know, yelling at the AI was pretty common. You would say like, Just go do this task for me, or, or assume you are an analyst that is helping me with marketing. And today people don't say that anymore. Uh, users have sort of like evolved how they use AI. And so when users know this is like the next generation frontier model, um, the way their expectations are much higher and also the way they prompt these models and the kind of tasks that they give them are also much more, uh, open-ended and complex. Seeing that is really fun.

One funny experience I had yesterday, and I wonder if you've had anything similar, and it does speak to the inadequacy of evals these days, is just I feel like sometimes you have to let a model now just run around in your space. It's almost like having someone over and really getting to know them in an environment. So I did that yesterday. You know, I've got all these kind of skills that help produce the podcast, and usually they just kind of run and do their thing. And Fable, out of nowhere, decided to pop up— the first time this has ever happened— a questionnaire for me about how I felt about an episode of the podcast it was about to produce and what my big takeaways were from it. And I didn't ask for this at all, but it, you know, it took initiative in a way that Really surprised me and made me feel like, oh, this is gonna be a very different kind of interaction. And the questions were, you know, were very good. Like, we've all had this experience of, oh, that output was really good, but this one was both, you know, I didn't ask for it at all and it was really good. So have you seen kind of surprises and is that something that you just kind of let users do and then tell you if interesting things happen? I guess it's potentially also a little bit hard, you know, given a harness and given a, you know, a certain bunch of product assumptions to even kind of allow those sort of emergent surprising moments to shine through. But how do you think about kind of capturing this spontaneity, initiative, proactivity, going above and beyond that models are now capable of that is really coming to all products everywhere at exactly the same time for the first time?

Absolutely. So I think you're spot on about The harness has to evolve with the models evolving, and it sometimes gets difficult to capture that spontaneity with a new model idiosyncrasy or behavior that emerges as these models get better. You have to evolve your harness to be able to do that. One of the things that we have focused on is building pieces of the harness that are going to be omnipresent. They have to exist for the model to be able to do more as opposed to getting in the model's way. And so for a lot of, for a lot of part of like the AI application cycle, what we have seen is people build parts of harness that guardrail the model to do very specific things. What we want to do is like get out of the model's way and give it the pieces that it would need to do the tasks that it doesn't have access to. For example, like public markets data or internet data, uh, or ability to run and execute code. We were pretty much the first AI startup to give language models their own code sandboxes before they were even called sandboxes. And so, um, soon Juice will have its own browser environment to go browse websites and private data sources. And so, um, things like that are— we're always going to keep in our harness. And in terms of like the new models, um, one of the things that I've seen our users talk about in our community and, and in our support when they reach out to us and tell us like, whoa, this is a pretty cool model, or like this where it's messing up is, um, 2 things. One is, um, they immediately when they try this new model, they give it tasks that they normally wouldn't have given to Opus 4.8 or GPT-5.5. Um, and, um, the expectations are much higher. And so, um, they, uh, they expect the model to be able to impress them right away.

Mm-hmm.

And, um, Because it's a new model release, there are some higher failure rates on Anthropic's side. And so those frustrations actually alleviate whatever pain the users have. And so I think we're a little early to tell on that front right now, but I'm excited to see how things play out in the next 24 to 48 hours.

Yeah, all the timelines are compressed. That is absolutely for sure. A question on economics. There's these competing trends. One which we've talked about earlier in the show is just the continued deflation with a notable difference between the original Mythos price and the current Fable price. So that's really favorable. Then there's this other big kind of price discrimination going on between your Claude Max account and your API usage. And as an application layer developer, you're obviously looking at the API rates. Maybe the best thing that could happen for all the app businesses right now would be for Anthropic to just say there's no Fable in your Max account anymore, or you can have it, of course, but you pay the API rates. Because then that would really, I think, put a lot more value on who has figured out how to get not just Fable, of course, but a whole orchestration of models with their different capability and price levels to like really excel in an area versus if I'm getting a 20-to-1 subsidy, you know, price advantage with my Max account, then a lot of the time it might just be like, yeah, it kind of make me the app out of nowhere. And, you know, it is smart enough in this day and age to like get over a lot of those humps on its own. So, you know, I don't think we're gonna see no fable in or no subsidy, but like How do you think about that difference? You know, what do you kind of hope happens? What do you think would be, you know, fair if you would want to go as far as to venture? Like, what, what would kind of lead to a healthy ecosystem where not, you know, we're not just having Fable recode everything for everybody infinitely, but also, and in that way, like sustain some actual diversity across the products and experiences that people have?

Yeah. So There's a couple of things that are happening. One of them is, yes, you do get some level of subsidy through the model providers if you have a subscription directly with them. And every time there's a new model release, kind of in the user community, people want to use the model a lot. And there's this expectation bar, and some models exceed that expectation bar. Some don't. But very soon, within weeks, there's like a sobering moment where people realize, okay, cool, like I'm done playing with the model. Now I actually need it to do useful things for me. And as long as, as an application layer product, you— your harness is built in a way that actually allows people to do those useful things in a reliable, repeatable fashion. they will want to come back and use that. And they would rather pay the API prices, whatever prices they need to pay to accomplish the tasks that matter to them. The second thing is the incentives of these model companies are kind of misaligned. Yes, they give you subsidies to, you know, on the tokens, but also they are incentivized to get you to spend more tokens. They are incentivized to get you to run through your max subscription usage as fast as possible so you can have a second, third, 4th, 5th max subscription. And so that's why you end up with like, um, you know, a loop of, of, uh, a loop that writes the prompts for your coding agents that then has nested sub-agents. And it's— there's going to be a sobering moment where people ask like, okay, is this actually, uh, a step function increase in my coding output, or, um, am I just token maxing right now, uh, as opposed to like results maxing. And so, so I think the correction will happen when there's a third player, and I think that's going to happen with xAI, um, when, you know, if the Cursor deal goes through, Cursor gets access to really— uh, xAI gets access to really, really good coding data and an incredibly good coding harness, um, and I, I— my bet is there will be a third frontier coding model, um, along besides, you know, Claude and, um, and OpenAI and, um, with Grok. Um, and when, when, and when you have a 3rd coding model, that's where it, it kind of increases competition on the, the, on the, on the, on the market. So that's our bet.

Um, When, as we move from kind of prompting to this kind of agents where you kind of give them the goal as you are the loop and you let them kind of run, what are the major differences you kind of see in the behavior as we've kind of transitioned in the last like 4 or 5 months, I'd say? And what do you see is this behavior like? What is this meta agent that you refer to? What is the difference in behavior?

So the difference in behavior is how you would treat working with a colleague versus a coworker, as opposed to a contractor. For a contractor, you have a super scoped out project and a super scoped out task. And all they have to do is go finish that task. For a colleague, you typically have a goal or a team that you're working with in a company. You set them a goal like, hey, this quarter we want to increase revenue by 40%. Or if you're a marketing team, it's like, hey, this quarter our goal is to increase top of funnel by 30%. Now how you do it is left up to the team. The team needs certain resources. They need a budget. They need a Maybe more headcount, all these different things, but the team has a goal or the coworker has a goal. And so the way I think about where we're headed is you define the goal, you don't define the how or the what. It's sort of like, this is the goal outcome you want. And then it's left up to the team or the coworker to go try a few different strategies to hit that goal. And so It's undeniable that we're going to go back from this. This is just kind of how knowledge work has always evolved is we, you know, you no longer run a copier machine, you have like very broad goals like, hey, we need to hit these business goals. And so that's why I'm pretty bullish on People starting businesses that run end-to-end with AI and people building teams that run end-to-end with AI, especially the new initiatives that they're rolling out. I mean, the lowest cost and the lowest downside thing you can do is have an AI go explore a thing for you. And, you know, 9 out of 10 times, even if there's like a 90% failure rate, you actually end up not spending as many resources on that problem. And so with being able to do things like parallelization, you can actually have multiple agents and multiple teams working on a few different goals and then increase your rate of success.

What about... You often have to hand over kind of credentials, right? Like, one of the things that I find working with agents is that the biggest friction is the credentialing. Like, you have to, you know, auth here or log in there, and you have multiple accounts, and you have multiple platforms that you are, you know, and sometimes I forget which platform I'm using which credential with. And like, I have this, like, where did I put— Where did I put the key for that one? Yeah. How do you think that can be managed better? I mean, is— do you just hand over your entire digital life to an agent to manage and then, you know, just run with that? Like, how does this work going forward?

I think that's the intermediate state. I don't think we have reached the final state for this. We're, I think, a few steps away from it. My bet is the way the future will look is most tools will, instead of giving your own credentials, most tools will allow you to invite AI as a collaborator with you, and that allows you to remove access for the AI without compromising your credentials. And the second thing is, I think the internet will need to evolve. To be able to serve agents as customers and build the toll roads for agents to traverse on. And so that's why I'm pretty bullish on agentic payments, where you know if an AI wants to go buy a product for you or buy buy a tool or try out a tool for you or try service for you, being able to give stablecoin-driven web-based payments for payment rails for the AI to use is going to be really, really powerful and is kind of where I think the future is headed. Because it's kind of undeniable that the next generation of users of the internet are going to be AIs. And so how do we evolve the payments railways to the infrastructure for the AIs to be able to navigate the internet?

Mm-hmm.

Buy services, buy goods for you, perform tasks for you. And so my bet is that more and more products will allow you to invite AIs as collaborators. Like, imagine a Google Doc where, uh, you know, ChatGPT, um, Codex, Claude, uh, Julius, Kimmy— all these tools are like collaborators, uh, or your Google Drive, right? Um, and you can add, remove access, you can see history. Um, the AIs can see what other AIs have changed. And then the payment system to let them go, you know, do tasks for you on the internet, microtransactions, and things like that.

I think Andrew Lee of Tasklet always just echoes in my mind with his statement that he thinks kind of everyone fundamentally is building the same thing. And another angle he has on it is only 3 kinds of software companies survive. There's the core intelligence provider, then there's the horizontal layer, and then there's people that sell outcomes. So I don't know if you would agree with that. I'd be interested in your kind of take, but it does seem like Julius and Tasklet and increasingly Zapier and Lindy and sort of all these things that kind of had a different form factor originally, are converging to be a general purpose digital assistant. And it's like basically you have some sort of history or some sort of point of view, but it seems like the differentiation is less over time, right? Everybody wants to kind of do everything. Everybody's building in all the tools. All of these things now have a browser increasingly. And it seems like part of that is because it's so easy to build these features in, right? And then obviously the The coding agents can help you do it, that it's kind of a, well, why not launch that as well? And we'll make our platform more well-rounded. I guess questions are like, is there any limit to that for you? Or do you just go all the way to the best possible general purpose digital assistant that you can provide? Go for it with that.

Yeah, totally. I think Andrew— is that the right name?

Mm-hmm.

I think Andrew, what Andrew said, there's definitely some truth to that where, yes, it is easier to build these new capabilities. I think teams need to be very opinionated on what capabilities they do want to add because the last thing you want to end up with is a lot of capabilities that don't cohesively work together really well. Um, what's gonna matter in the horizontal layer is even when you have a horizontal layer and a general-purpose agent, people are going to have preferences with who they work with and what they work with. Um, and so a simple example of this is like, um, what do you use let's say your general purpose agent has access to financial data. Do you have access to real-time financial data? Do you have access to stale financial data? If it's real-time financial data, you will be charged very differently because you have to pay these stock agencies, stock exchanges to use real-time financial data versus stale financial data. And so those kinds of like opinionated bets are going to matter. Now, to what extent is TBD. Um, I think there's some truth to what he said, but what's going to be really important is do users prefer working with whatever opinionated bet that you're making versus some other companies making? Um, typically, like, you know, I think of this as like when you be— when you work with— when you interview somebody for a job and they Clear all the technical rounds, all the hard skills, but then you have like three great candidates in one role. You have to decide, hey, who who is the best cultural fit for us as a team? And I think that's kind of what's going to happen with these horizontal layer agents is who who do I personally like working with the most? Whose opinions do I prefer and bias towards the most? And who I see as like the right fit in my stack? And I think that's going to be increasingly more important. For example, do you prefer an AI that produces artifacts for you as a final result, or an AI that, um, you know, gets you more involved in all the intermediary steps? I think those seem like simple decisions but actually do matter a ton in terms of like how people want to work with AI. And so I think those kind of opinion bets are going to matter a lot.

Yeah, that's interesting. One, one quick follow-up and then I'll give it to you, Prakash. The, um, the other big paradigm I wonder if it's like a way to think about what apps should do is try to make themselves a winning option to be hired by the core agent, whatever that may be, that individuals have in the future. So can you get to the point where Claude comes to you because you can do something better, cheaper, faster, higher fidelity, better access to data, whatever? So how much do you think MCP API, A2A, whatever? I don't think that form factor Tell me if you think differently, but I don't think that's going to matter that much how that's done, and more is going to matter, can you offer a service that Claude would rather buy than build? How much do you think about shaping Julius to present in that way?

Certainly. I think that's going to be, and that's why I'm really bullish on agentic payments where Julius and Claude should be able to transact with each other on behalf of the user. to be able to accomplish tasks. And Julia should have a reputation that Claude and Codex and Grok and other AI agents can verify and validate. And so there is a future that there exists like a Yelp or a Gartner for AI products that other AIs maintain and regulate. And so I think The one bet you do not want to make is the internet will not be run by AI agents. Agents will be first-class citizens of the internet. The other bet you don't want to make is humans are going to talk day to day more to AIs than to other humans. And this is not in a dystopian way. What I mean is just think of in the work sense.

Yeah.

You know, 8 hours in an 8-hour workday, you would before you would talk a ton to your human colleagues, and now people just sort of they're whispering to their computers and they're propping their coding agents, and AI is reviewing their code, and like you're you're going back and forth with AI. And so so so those are the bets you don't don't want to make. You want to make the opposite bets, which is the the internet the agents will be first-class citizens of the internet, and then humans are going to talk to. AIs than other humans day to day. And so given that, it's kind of undeniable that there will be a future where agents will transact with each other and hire each other for tasks. And it's just probably better for Claude to hire Julius for a data task than to build something very opinionated on its own, not because it's not possible for Claude. It's just like, it's more cost efficient. You know, when you want to build a shed in your backyard, you just want to hire a contractor, not do it yourself, even though you hypothetically could do it yourself.

I sometimes call that return on compute, because if an external agent can save you compute, then you should use that external agent. So an agent should use a search engine rather than indexing the entire web because it's cheaper to use a search engine which has already done the precomputation than it is to build your own index. So in every case, it's a, it's a return on compute question. Uh, just a segue here. Um, so we've been using Fable. Uh, we've come across, you know, people posting about rejections. Uh, in, in my, in my tests Almost consistently, whenever I tried to address the production database or the production site, Fable would drop off to Opus 4.8. I believe your users in Julius are using Fable through the API, and you are also very heavy on data science users. And one of the aspects of, you know, work that Fable is banned from doing is machine learning work. So how have you seen The rejection rate on your platform? Does the API work the same way in the sense that it drops off to Opus 4.8 and then it gives you a rejection message? How does that work?

Yeah, so we have seen failure rates on tasks that involve really advanced coding, that involves, you know, write me, you know, use scikit-learn to like perform, you know, train this model. But we haven't seen failure rates on other kinds of data tasks. For example, like, hey, I want to start a landscaping business and like, can you help prospect leads for me? We will see failure rates for where it will sort of trigger safety filters where it's for things like you're prospecting leads for a landscaping business and the AI says, oh, this is personal data. even though it's publicly available on the internet. You know, let's say, you know, Prakash has a— Prakash's Landscaping in Philly, and there's contact information. It is kind of borderline personal data, even though it's available on the internet. And so I think— so that's kind of what we have seen. I believe it doesn't fall back to OPIS. It's just like a failure in the API.

Interesting. So the fallback to Opus is a harness thing on Claude, on Claude, on the Claude, you know, frontend then. That's interesting to hear.

Oh, I can hear you.

Nathan, I think you're muted.

Sorry, got some background noise here. I keep doing that.

Poor form.

Thank you for being here, Rahul. Great to meet you today. Any closing thoughts you would wanna leave people with about what you're watching most closely, what you're like most excited to unlock next? You know, what's the alpha that people should be watching your space for?

A big alpha is the— this is probably the greatest time to start companies and start businesses with the AI, with the help of AI. You know, there's no reason someone shouldn't have a business that they've been meaning to start for a while. Um, they shouldn't be trying— they should— there's no reason they should not be trying that with AI today. I mean, if you, if you won't, you're probably going to be left behind. Um, and the cost of trying new business ideas is like, uh, is at an all-time low. And so, um, I would say like, you know, this is, this is, this is the greatest switch. You know, there's the dystopian view of the future where it's like AI is going to do everything and we have nothing to do and And then the other optimistic view that I personally have is, wow, this is, this is finally an opportunity for people to go build and start new businesses, start new products. And you should totally do that today.

Yeah.

Indeed.

Love it.

The positive vision for the solopreneur future. We definitely need all the positive vision we can get. Thanks for being with us. We look forward to following your progress and talking to you again before too long.

Thank you guys, have a good one.

Cheers, Raul. Awesome.

Yeah, it's good to have a little positive vision note to end on. Um, my hair has been sufficiently blown back again that I'm kind of in a philosophical and somewhat somber mood, I would say. But you know, that's mostly because I do take the upside, not for granted, certainly, but sort of as something that I think we're gonna get if we can just manage to keep our heads on straight these next couple of years. And yeah, there really is a great potential. You're starting to see this sort of vision of like mass customization, just infinite worlds to explore.

Right?

I mean, when I was saying at the beginning that the— what had struck me most was the scope of what the, you know, even just a couple prompts with, um, Fable can often do. It does sort of suggest this, this future. Again, we got a lot of work, I think, to do to, to get there, um, but where we really do have so much, uh, in the way of adventure experience that can be created so quickly and curated so well by people around us. The flourishing of kind of digital experiential culture might really be hitting an inflection point in the immediate future. And we kind of teased around this a couple of times, but the robotic future is not far behind. It is interesting, and it is a deliberate sort of slowing of the Dyson sphere constructed by humanoid future that they are blocking the ML use cases as much as they are. But internally with trusted partners, we're going to see, I think, this massive spectrum of access develop. And even though you can't go necessarily do it in your mainline Claude account today, the acceleration that this implies for robotics, I think, is also just dramatic. Later this afternoon, too, I'm talking to one of the cofounders of Neural Concept for the cognitive revolution and just engineering of all kinds of things. These things are going to get— I think with Fabo, they probably are getting good enough to do a really significant part of the physical engineering of the humanoid forms themselves.

Right.

the components of those. So the feedback loop is strongest, of course, in the digital sphere, but I really think we shouldn't have any doubt at this point that robotics is going to work, and pretty soon. You know, there's no way that the optimization that a model like Fable can do is, given where we are in robotics, is going to fail to deliver pretty darn good robots in the not too distant future.

I think it's interesting that when you look at the timeline, you can start to see this kind of single-line timeline kind of go through a gas chromatograph and kind of spread. And now you're seeing the spread and you have 2 months ago, you had the government get access to Mythos level. And then you have basically power users who are able to pay $200 a month kind of get access to that Mythos level 2 months later. And you can kind of see 2 to maybe 4 months, 5 months, I don't know when, you will probably see the average paying user paying like $20 a month, get access to Fable or Mythos level. And then you kind of see that maybe like a year later, the average free user kind of getting access to that same level of intelligence. So you're starting to see this kind of gas chromatograph scattering of, uh, when people get access depending on how much they pay and how much utility they have for the product itself. Um, and so everyone kind of gets there eventually, But some people get there first depending on whether they have a lot of utility for the product. And I guess the hope with having 2 or 3 firms in there is that the spread between the people at the frontier getting access early to the people at the very end getting access for free is not that large, right? And to note, there's another set of people who have access even a couple of months before that, and you have to belong to a lab. So if you belong to a lab, you get access maybe a month or 2 months before the government itself. Then you have the government, then you have enterprise, then you have power users, and then you have normal paid users, and then you have the free users. So you have this kind of spread, this gas chromatograph kind of spread of when people get access depending on how much utility they have.

Yeah. And that does lead me back to this kind of unpleasant conclusion that watching the frontier companies super closely is, I think, a pretty valuable activity. There's going to be a lot of secrecy around this, and that just means public analysis is more And more necessary. Yeah, I'd love to be going and, you know, doing all these different Julia demos and exploring the virtual worlds that Fable can create. But it seems like we're going to be kind of to a very significant degree until— it's hard to really imagine a change to the regime that could change this at this point. Because even if the government gets much more involved, it's still going to be getting much more involved in regulating these few key players. You would have to have a pretty radical change to the landscape, I think, from where we are today to not have a lot of gravity pulling you into just paying attention to, watching, trying to make sense of, trying to influence how many theories of change right now ultimately have to run through getting the frontier companies to do something different. That really is a striking volume of where the value is, I think. It's a strange reality, but it does make for a sort of TV-sized cast of characters somehow, which is interesting metaphysically.

I, I do see kind of a failure of the companies to prove utility. So you can kind of see that in the sense that, yes, you know, we on the— we are the power users, we can, we can kind of see the use cases and we can kind of think about this. But you can see in the market every couple of weeks there's a, you know, AI has failed, AI has plateaued, uh, and, and primarily because for the average user, the average user is not using this to solve Erdős equations. And so the utility is not being proven out for the public at large and for many enterprises at large at this point. So it's really the companies have this ramp task. You can ramp your research, but that doesn't— the product, but you haven't proven out the utility to a lot of people. And that means that to some extent, we actually have a timeline which is slower than it could be, which is perhaps a safety, a pro-safety thing in the sense that it's only going to move as fast as the humans can absorb it. If the humans can't absorb it, they're not going to use it. If they don't use it, it doesn't get— we don't get the next generation. So on the bright side, that means that you know, maybe the safety is happening as it, as it should. Humans absorb it at the rate that they can, right? So at the rate that they're willing to pay for, right?

I predicted dramatic acceleration, though. I mean, if there's a, if there's a bet to be made there, I think I'm gonna take the other side of it. Because this really does feel like another critical threshold. I have this podcast coming out today on the Cognitive Revolution feed with Rebecca Hines from Glean.

Yeah.

They're putting out this report on their new work AI index, which is results of a huge survey and all this synthesis. And they're introducing 2 new terms, botsitting and botshitting. These are the 2 behaviors that are sort of good but frustrating and just outright bad, at least so far, right? Like, bot sitting is you're providing all the context. And of course, Glean is in the business of hooking up context to models.

Yeah.

But I think they're absolutely right that people working in fragmented environments at their— I was just talking to a friend over the weekend about this who works at a giant company. He's in-house counsel there, and they've got legacy systems going back decades, and they're kind of doing a roll-up strategy themselves. So he's got all these transactions, legacy systems of all the companies they're buying, and the fragmentation could not be worse. So people are kind of sitting around doing this bot sitting type of work of copy paste, whatever, and they're getting frustrated by it. And then when they get sufficiently frustrated, some of them tip over into bot shitting, which is just like passing off work that you yourself do not understand. and cannot explain or defend.

Right.

And surprisingly, like, shockingly, honestly, to me, 69%, I think it was, of people in their survey admitted to doing that, to basically sending some down the line, right? Some work product artifact output of an AI that they couldn't explain or defend if asked.

Yeah.

What I think is gonna be different though this time, and why I think the demand is gonna be there in my view, is like, I think that the Fable class models are going to actually just do a lot of people's jobs well. You know, where like it was a problem before when you were botchitting because they didn't have the context, they weren't really there.

Yeah.

But as the context comes online, and again, another, another big advantage I, I'm detecting a little bit with Fable is it's just really good at searching through the context even better than before. And they were already getting really good at that. So as long as the pipes are connected at all, you know, it's gonna be really good at kind of exploring them. And then I'm seeing this sort of dramatic closing of the gap between the things I used to have it draft for me and how interesting I thought they were, how much it anticipated my angle on them. That gap has closed so dramatically. So I do think this has— and there will be compute limits, of course, there will be cultural limits, but increasingly, I think I would expect the word to get out pretty fast. that if you just kind of plug in Fable, it can actually do a pretty good impersonation of you. And in a lot of contexts, you know, that's going to be like really, really appealing for a lot of people. So in that sense, I really do think we're headed for a pretty— I think it could be a pretty dramatic reaction.

One of my predictions has been that A lot of people have already prepared access to context through OpenCLAW setups. So you probably have between 10 and 30 million people worldwide now who have OpenCLAW or Ermes agent setups where they've already provided access to Gmail and calendars and research and their own, and basically their entire computer terminal. Because these things can act on the entire computer terminal. And the question for me has been, are we just going to see a model upgrade that happens underneath where you move from one generation of model to another, and all of a sudden these things can do everything you can because they already have all the access that they need? And so the way that you present it, maybe this could be it, the fabled class of models when they get cheap enough that people put them into these OpenCLaws and Ermes agents. That's pretty much it. All the affordances are there. They can write email, they have access to your computer, and they can just do it.

Yeah, I've described myself as precious for a long time. One of my good friends in the AI space, who I think is from whom I've learned the most in a practical, applied way, my friend Chris York, who has a small internet profile, but he's out there. He has just been first of all, extremely good over time at being a very clear thinker, a systems kind of guy, somebody who can really articulate what a good standard operating procedure should be for AI. And he's been extremely well rewarded for that ability by the great service that AIs have given him in response. The other big insight that he's had that I really try to keep in mind is For many, many things, it really doesn't matter that much. And you are just being precious. And so updating our own mental models of like, where does it matter and where do I still want to be precious is going to be, I think, a real, you know, a real introspective kind of growth moment, you know, for a lot of people because it wasn't— for me, it was still like a little bit, eh, you know, I can't help but be precious.

Mm-hmm.

With this new thing, I think it's gonna be a little bit more of a dance, you know, a lot more of a dance potentially of like how much of the stuff that I used to feel like only I would do something if it was coming out in my name, for example. I have actually, I don't know if I, we didn't even talk about this, but I, I'm doing a Fable takeover of my Twitter account today.

Mm-hmm.

I figured, you know, let's live in the future a little bit and get that run in this morning, you know, and, and make good on, like I, I've said many times, right? I'm, I know I'll, I'm winning with AI if I can spend more time outside, get more exercise, you know, invest in my health and have the AI, you know, keep me on the rails at the same time. So to just kind of explore that, In a way where I think it suddenly is like probably gonna tweet just about as well as I'm going to, you know, when it comes to putting things out for today's show and getting the— I didn't even, you know, I gave it a total green light. It was able to schedule its own stuff, you know, find the tags for people. Um, will it make a mistake? I bet there'll be a mistake in there. I usually make at least one, um, you know, over a handful of tweets anyway. I decided to flip this switch. Not that I don't think I'm gonna keep it that way. I don't think I'm gonna like give Fable my Twitter account forever, but it was kind of a sort of exposure therapy for myself in terms of, okay, now we actually, we are getting to the point where the preciousness is gonna start to work against you. Preciousness was a great shield against bot shitting in the past.

Mm-hmm.

I never wanted anybody to think I was just passing off AI outputs to them.

Mm-hmm.

But now I'm gonna have to be like, what is the hybrid form? What is the winning recipe? Do I start to sign these things? You know, like by Claude under Nathan's direction, you know, Fable being Fable. Um, how, you know, it's, it, it is going to be a, a whole new space to explore.

Mm-hmm.

That, um, it's gonna be very, very interesting, very productive, very exciting. Very, very challenging, I think, for a lot of people. But it's definitely happening now as far as I can tell.

Welcome to the future.

So tomorrow we'll see what happens. We've got Fable working in the background. The experiment here is can we do a Fable show and tell with a bunch of projects that people have already made in the first 24 to 48 hours?

Yeah.

And there's plenty of candidates. Fable had no trouble going out and sourcing a bunch of interesting projects to talk about. It's also got the DM access for me on Twitter now. So, we're sending out some DMs. We'll see if we can get people to come and actually talk. Response hasn't been a rush to the calendar just yet. So, I don't know if people are And my— I did instruct Fable to be, uh, upfront about who it is rather than, you know, uh, leaving people even temporarily to think that it was me typing these messages out by hand. Uh, so we'll see what we have.

Yeah.

But the sort of experimental gonzo nature of tomorrow's show should be pretty interesting, uh, whether we have any human guests or we just have to have Fable serve as our guide to all these, uh, new Fable created worlds, it's going to be interesting one way or another.

Indeed. And on that note, we will see you tomorrow.

Thanks, Prakash. See you tomorrow.