AI:AM — RSI Gets Real, the Context Bet, and the Benchmark Anthropic Fails · June 12, 2026

Good morning. It is Friday, June 12th, 9:12 AM. Welcome to AI in the AM.

Good morning, Prakash. It's another exciting day in the AI sprint at marathon length all the way to the singularity.

Indeed. And what, what has been your overnight update been, Nathan?

Well, I think we're starting to see, you know, more and more thoughtful uses of Fable and more reports on what's happening with it. And as always, these things kind of mature over a few days into, you know, a more sophisticated understanding. First 24 hours, it was just like, I gave it this one prompt, it went and built this whole virtual world. Isn't this crazy? And lots of interesting examples of that, a little bit of which we covered yesterday. I do think I'm still in the process of trying to recalibrate what a fellow Nathan— Nate Jones, I think he goes by most of the time on TikTok and other short-form platforms— calls task imagination. Basically, what are you gonna do? What are you gonna ask Fable to do that is actually up to the scale of its capability. He, I thought, gave a great little riff on this the other day saying like, you've probably never done anything that took AI an hour to do. Now this thing can run for a couple days. What are you gonna give it to do? Everybody needs to recalibrate and really expand their minds when it comes to the scale and scope of their task imagination. So I think that's one thing that I'm still working on. One of the more kind of differentiated things I do is write outlines of questions for podcast guests. And I was working with Fable last night on a couple upcoming episodes, one with an author. And I usually don't do too many episodes about a book, but this one is about an upcoming book. So I had listened to the book as an audiobook. Naturally, I used AI tools to kind of extract a clean transcript out of the PDF preview that I got, get rid of all those, you know, pesky page numbers and chapter titles, whatever that you would have in the script if you just did a naive OCR, then put it over into the ElevenLabs reader to read the thing to me. And so, you know, I did put the time in, but then When it comes time to write the— sit down and write out the outline of questions, I don't have at my command every little aspect of the book, of course, right? So I'm not taking margin notes as I go, as maybe I should be. So I put the same version of the book into Fable and said, give me— look at my old stuff, of course, and give me your version of this outline. And I again was super impressed and it really did reinforce this sense of a sort of new way of working where I do need to be open to a hybrid output format. You know, it is not the case, I don't think anymore, that it really makes sense to try to rewrite every word or claim every word as my own. It gave me an absolute ton of stuff, like way more. cover in an hour. So as I refine the skill, one thing will be, you know, definitely getting it to just be a little bit shorter and, you know, more suitable for the time that we'll actually have. But it just did such an incredible job. I thought the taste factor was so high in picking out quotes from the book that motivate interest, what I think will be like a really interesting discussion. A lot of these ideas I sort of had floating around in my head and I think I could have got them down on paper, probably with a similar amount of time spent that I actually, you know, put in interacting with the Fable output. And I do think it's still going to be super important if I'm going to show up for a conversation. I've got to, you know, got to do the work to be ready for it in my own brain. That can't be fully externalized, I don't think, as long as I'm the one having the conversation. But it definitely took my Prep to another level. And I think the— my ability to go into this conversation and cite passages from the book that are— that were really extremely compelling, you know, little turns of phrase or analogies that the author had made, it's going to allow me to be, I think, more concise in my presentation, which is, as you could tell from this monologue, not a great strength of mine. And, and really kind of tee up the author in a way that I don't think I otherwise would've been able to do. So this, this sort of hybrid recalibration task scale and scope reimagination, I think is one of the biggest takeaways. And, you know, at the same time, starting to think a little bit more, I haven't hit my weekly limit yet. But I'm watching it and, you know, we're, we're probably gonna hit it. So thinking a little bit more about how do I, in some cases, probably just pay overages. I do think it's gonna still be worth it in a, in a lot of cases. You know, even if it was $20 to get that outline done, to buy back the time and to do a better job, worth it. You know, I, I think the—

Yeah.

I predict that the willingness to pay is gonna be very high for Fable, and especially with some future update to data retention policies, I just expect their revenue to continue to go vertical with this thing. But also started to put in some strategies yesterday for especially for coding. Fable write the plan and then coordinate other coding entities, specifically Codex, which I mentioned yesterday. I have, you know, if we believe SemiAnalysis, I have $14,000 worth of credits that I'm really, really underutilizing.

Right.

And perhaps undervaluing because I've only been paying the $200 a month for. So I'm gonna try to get Fable to, in a way, sort of help me with my task imagination scale and scope, but also have it start to do more delegation and checking of work to Codex. And hopefully that will allow me to get a lot more leverage on what, uh, token budget I do have included in my max plan.

So one question that I had for you is that I think, I think you're one of the people that sees very clear improvements, uh, between GPT-5.5 and Fable. And I find myself rather confused because many of the— on some metrics, on some benchmarks, Fable outperforms by a very large margin. But you also have other benchmarks where you have Fable with a 77 and GPT-5.5 with like a 76. There are many benchmarks where you have them very close and some where even GPT-5.5 is ahead. And then there are some category where it's above and beyond anything that's ever been seen before. How do you assess on a vibes basis, obviously, because we've all seen all the benchmarks at this point. How do you assess on a vibes basis that it is truly better?

Yeah, I mean, it is vibes to a very significant degree. I'm reminded of a great Dean Ball passage in response to Claude Opus 3, where he asked it what music it would want to hear if it could hear. And it came back with, uh, if I recall correctly, a Beethoven piece, which was rich with meaning. And, you know, how much Dean sort of layered on himself versus, you know, how much, um, Claude III Opus brought to the table on its own, you know, it's probably a little bit subject to interpretation, but one of the great ironies of it was that the piece that Opus wanted to hear was a piece that Beethoven had composed while going deaf and sort of never really heard it himself.

Mm-hmm.

And so after, you know, doing a close reading of all this Opus stuff, Dean finally said, here finally is an AI whose thoughts I want to hear. And obviously, you know, that was 2 full integer versions ago.

Hmm.

But I feel like there's something similar there where it's like, here is an AI that I actually want to work with. I don't feel like I am just having it preprocess data into a more useful format. Or, you know, it's— I've got a lot of value over time from collecting links, you know, just giving me a big bunch of links to help me prep for an upcoming conversation. You know, that's extremely useful from a time-saving standpoint. But I have never considered a previous model to be like up here in a way that this one does kind of feel like up here. It feels like I'm— you know, to say it feels like a different version of myself is too strong. Um, I don't want to be— I don't want to merge with it that much. You know, I still, still want to, I think, retain some healthy boundaries between, uh, my cognition and its cognition, but it's— some of its insights and some of the ways that it was phrasing these questions just felt extra incisive, extra insightful. You know, there was something about it that is hard to put your finger on, but it was like, oh, that's really good, you know? And if I had come up with that, I would have been pleased with myself. Maybe that's one way to think about it, is just how often do you have this sense That if I had had that idea, I would have felt pretty clever and satisfied with myself for, uh, for how good of an idea it was. I don't— I'm not tracking that in a quantitative way, but it feels much more frequent with this new model.

Mm-hmm.

And I'm keep— again, I'm keeping a lot more of its Mm-hmm. I used to really rewrite every word, and now I'm like, well, there's a couple things here I might change the framing of a little bit, um, but I'm catching myself in the habit of rewriting every word. But also increasingly I'm like, I'm not actually making it better by doing this. I used to feel like I was legitimately, you know, undeniably in my own mind at least, making it better. And so the discipline of sitting there and rewriting everything felt not just like ritualistic or like something that, you know, I was doing to really own the output, but something that was actually making the output better. And a couple times this week I've found myself still going through that and then just being like, no, it's not really any better. And then trying to kind of, you know, when that happens, at least for now, and I think I really have a lot to— it's gonna take some time to settle into a new balance and know what I actually feel good about.

Hmm.

Because I do think there's something antisocial, even though I've been experimenting with it a little bit on Twitter over the last couple days, there's something a little antisocial about just spamming the world with raw Fable outputs, um, yeah, you know, in an unrestrained way. So I don't— I definitely don't think that's where I'll land or where people in general should land.

Yeah.

But when I've felt this feeling, I've kind of purposefully said, okay, well, let's go with its language for the time being rather than just redoing it purely to redo it. I feel like that I need to get out of a mode of being so precious as to feel like I needed to type every word. And where that settles, I don't yet know, but it's clearly going to be, I think, in a different place than it was before.

I, I think like we are in the stages of this kind of collaboration with AI, right? And I think it's like a two-way street, kind of. The AI goes like some part of the way, but you also have to engage. And I think every person has like a different level of trust and confidence that the AI has to kind of meet. And it strikes me that some people are Have already kind of, I'm just gonna go with the flow, like, just let it go, right? And, and they have their AIs like managing their email, managing, managing, you know, all of these things, even with like very inferior ones from 6 months ago. I wonder, I wonder if their results might not be better than the people who are used to micromanaging and making sure everything kind of works. When the AIs really do get better. It strikes me as a little bit of a counter the last 20, 30, 50 years of history where people who were more kind of management-focused and making sure things worked had an advantage versus the people who kind of just like, you know what, just let the world come at you. Let it go. It strikes me as a moment where we see a reversal of that. So it's interesting to watch.

Yeah, we talked about this a little bit yesterday, but I am definitely also thinking about this sort of bot sitting and bot shitting dynamic that has been happening at companies. And I really do feel like in terms of a mechanism to major economic disruption, if I'm a CEO and I'm just focused on my competitiveness and survival, I really think one interesting strategy that people might want to pursue is just really let it rip with Fable. Let everybody have access, encourage them to use it aggressively, you know, try to set the culture that, yeah, this is the new way of working, blah, blah, blah. We've heard obviously tons of that. And then somehow log everything on your side and then do some follow-on checks later with Pangram Labs or, you know, even just like Systematic, you know, checks of like, okay, who is actually passing off work into other parts of the organization with no or minimal changes? And then as you see that and you're like, the house is not on fire as a result of people passing off Fable work as their own, then you have a pretty good sense. You've kind of like mapped your organization from the inside a little bit. To know what jobs can AI do here.

Mm-hmm. I actually think that was the whole point of the token leaderboards earlier in the year. The token leaderboards, I think this is when every one of these large firms gave their employees a kind of like, you know, we're going to have a leaderboard, who uses the most tokens. If you don't use enough tokens, you're going to get fired, etc., etc., etc. And it was, you know, people were laughing about it on the outside. They were like, Meta is so stupid. Zuck is so stupid, like, et cetera, et cetera. And I actually think they were not. I actually think the CEOs were— it's very different when you have token anxiety. Like token anxiety is a big— is a, like, you don't try these tasks, which might take a lot of tokens and you don't try these tasks, which have higher probability of failure. And you end up in this micromanagement loop where you're like, I'm only going to assign you tasks that I know you can complete within the timeframe that you, that, that, you know, that is allocated to you with the success rate that I want. Right. So you only, you end up with this token anxiety thing and then you, you end up not utilizing the, or not trying the AI to the, to the extent that it should be tried. And I think what ends up happening is that When you have this token anxiety lifted, you end up assigning more tasks, more difficult tasks, and you're willing to accept a higher probability of failure. And you're willing to kind of like, you know, maybe spin up 4 different ways of doing the same task and just running them all and seeing what happens rather than just like sticking to your, you know, because You were always concerned with that, the number of tokens, and also the time that you have to spend evaluating. If you decided, look, I'm not going to care about the tokens and I'll just reject you the moment you get anything wrong, and I'm not going to bother to check that much. And then you basically are forced, or you force yourself to kind of use them more. And that kind of expands the scope of the capability that you're willing to you know, use from them. And I think that is what essentially the token leaderboards did. And it was very successful. It was enormously successful, I think, within the firms, within like Meta and other firms. I think it was also enormously successful for, you know, the sales teams on the AI labs, which is also why they are now doing this kind of, you know, we're going to release the limits and like we're going to double your limits. We're going to allow you Fable and we're going to And the reason, I think, is because token anxiety holds people back from exploring the edges of the capability. And that is really what I think the labs are trying to do at this point, which is why it's a little bit like addicting people. Because once you realize that these AIs can do certain tasks, you then start to evaluate, like, how much time do I spend doing this task on my own? And was it really a fruitful use of my time, or should I just have used the AI, which I now know can do these things, right? And I think that is really where we are at this stage. The models are capable enough, but we aren't handing them enough responsibility for various reasons, including we don't want to spend our time evaluating, and we have token anxiety, and we don't assign lower probability tasks, and a bunch of these things.

Right.

So, and I think, you know, that's the battle that the labs have to fight because the capability surface is not well mapped and they need people to map— every person needs to map it for themselves. So.

Yeah. You know, I was just thinking there's this sort of inside-out kind of hydraulic mapping that companies can do where they can just say, let's just try Fable in this role. It's almost like the old Amazon Web Services chaos monkey a little bit. I think it would be an interesting exercise for a lot of organizations. And this is, you know, again, the social barriers will be the biggest challenge here, but that's why maybe just letting people do it on their own and then trying to detect where it's happening rather than going in and doing it in a top-down, you know, mechanistic chaos monkey sort of way might give better results. But I think you could do a chaos monkey strategy in a lot of organizations where you could basically go to a role and say, we're just gonna have Fable do this role for a little while and we're gonna see what happens. We're gonna see if anything breaks. You get a vacation. Great news. Paid vacation for you for a couple weeks while we have Fable do your job. I really think we are getting to the point where that would work in a lot of places. You know, in my Twitter account takeover, Over the last 48 hours. It hasn't embarrassed me. You know, it's, um, it's posted not exactly what I would post. It's probably been a little— I honestly feel like my own persona needs to get more out of distribution if I want to be like actually successful online. Uh, you've, you've been much more experimental with your online writing and persona over time than I have. I've been very this probably needs to change because I think 3, 4 years ago when I was first kind of writing publicly about AI, my assessment of the situation was, I think rightly, everybody's asleep on this. People need good sources of really reliable information. And I felt like I could be, for many people that weren't paying enough attention, a really reliable source. And so I wanted to maintain my credibility above everything. And I wanted to be like, this guy has never steered me wrong. And I think that probably was, again, in a '22, '23 timeframe as people were just getting calibrated to GPT-4, which I had a 6-month head start on calibrating myself to. That may have been the right strategy. Now I probably need to get a little weirder, both because there's a lot of good sources of just basic factual information. that I can, um, you know, I can share the burden with. And, and also because it's all getting so hard to assess. Back then, you know, I was like still better than the models at pretty much everything, um, with maybe a few exceptions. Now it's like they're better than me at everything with few exceptions. So to be a little more speculative and kind of, you know, Mm-hmm. exploratory in one's analysis and persona is probably going to be higher return anyway. And another reason, of course, is because as everybody sort of does these fable takeovers, that's going to be just the sort of baseline information environment. And to be different is probably going to be necessary to be rewarded. So I think it hasn't done that. It hasn't gone out and been way more creative route of distribution than I have, but it's really fallen, you know, pretty well in line. And if I had been, you know, having a— so this was all me pretty much in the past, but if it had been a social media manager, I would kind of say, yeah, like, you know, Fable can do this job. I don't think it's gonna embarrass the brand. I don't think it's going to, you know, make errors at a super normal rate that would cause any trouble. And it probably, especially to do something like post a handful of tweets a day, we're probably talking still an order of magnitude, if not more, cost reduction relative to what a person would cost. The more I have meditated on the pricing, the more I'm like, yeah, it's high compared to tokens that we are used to, But it's also pretty low still on a task-to-task, head-to-head comparison with humans almost everywhere. Because it really doesn't take that many tokens to do a little Twitter research and put together a few tweets.

Mm-hmm.

Uh, that may take, you know, a person may do what, you know, a tweet an hour or 2 tweets an hour throughout their day. You can get that stuff done In not that many tokens, you know, $20 worth of tokens is, I would say, plenty to run your social media account for the day.

I think, I think, you know, I went through phases of, phases of social media. You know, there was a year that I did like a couple hundred million views on Twitter, which is insane. But basically what ends up happening on social media is that I think it is possible at this point using Fable To build a good social media manager. It's just that it would take a lot of tokens. It's still going to take a lot of tokens. And I think that that's fine, actually. I always say like, the AGI is here, it's just going to take a lot more tokens than you think it is. One of the things—

So where do you think, where are all those extra tokens coming in? I mean, I'm doing pretty mundane stuff where I'm just like, you know, here's our prep list for today's show, you know, put a few tweets together. It's not taking that many tokens, it will, you know, but again, I'm not like really trying to push the boundaries. Where do you, for what kind of results would you think that you're gonna need a lot of tokens?

So, and this is essentially what I do in my head all the time, which I've tried to automate a little bit, but it's very difficult. What has worked for me is, You have to be like Twitter. The timeline does not have a memory. And because it doesn't have a memory, it doesn't quite remember what has happened before. But people on the timeline have memory. And so what you need to do is you need to echo. And so a lot of what I've done has been to be the memory for the timeline where that memory does not exist. So I will point out things that happened like a year or 2 years ago, which have relevance to the particular moment now. I used to have a much better memory, not so much right now, but I have a very good visual memory. And so I can remember a graph from archive paper from 4 years ago, and I can at the right moment drop it in. That is definitely doable by a model. But the context, I don't know how much context and memory search you're going to need in order to do that. And I feel it has to go to a model because I'm not that good. I can't remember that many things, and even I'm okay. So a model which can actually have that infinite memory of every graph that it's seen and have also a ranking of what is actually proven or more accepted More relevant, and then injecting that into the particular zeitgeist at that moment. So the other part of that social media posting, I always compare it to surfing. You don't generate the wave. You're just one surfer there. You don't— you're not generating the interest wave. You are catching an interest wave that is building. So you have to watch and wait to see where the interest wave is building. And then you kind of paddle Really hard and you catch the wave. And when you catch the wave and you're surfing, people are like, oh my gosh, and they think you're doing it. You're not doing it. It's the interest wave that other people have in that particular topic, which is really driving, you know, the views and, you know, what's happening. And this is basically what every single person who does media eventually kind of finds out, that, you know, it's a combination of what you're good at and what you can talk about and the particular moment in time where there's interest in that topic, that, that builds. Um, and I think the models will get eventually very good at this because they are already pretty good at writing. They need a little bit more on like visual memes and like being able to edit video, but they have infinite memory. The infinite memory is there, right? Uh, the embedding— you can, you can do an embedding vector search and you can find stuff from the past, and then you can have ranking algos that Rank how important certain things are. So I think eventually the models will be better than us at social media posting, much, much better. And which has consequences because it'll be very interesting to read, but I'm not sure how comfortable we are going to be. It's all going to be model generated, but it's going to be fun and it's going to be awesome and it's going to be interesting to read.

So, Yeah. Cutoff date. I was just looking up the Fable cutoff date is still January 2026. So, that does kind of stack the deck against it a little bit in terms of keeping up with the zeitgeist or, you know, whatever the trends, right?

Yeah.

I do think the new Twitter API is a work of real genius in this respect because at— I think they've, they've done a great job of giving you all the access you want at a price that is like, for any given marginal access, not that much.

Yeah.

But which is still, for many people, especially as they bring agents up to handle a lot of this stuff for them, it's gonna just end up spending a dollar a day or whatever on the Twitter API.

Right.

And you won't really notice it, but, you know, that's gonna very quickly they could get to Facebook or beyond level revenue numbers just through that API as people start to scale agent usage.

Yeah.

So I think that's just yet one more reason. We haven't touched on the SpaceX AI IPO yet, but that's one more minor reason to be bullish on what they might accomplish.

But let's— have they— are they— are they live yet? I've been— I've been watching the, uh, I've been watching the IPO.

Um, I just saw a picture from Nikita who said, uh, always check your DMs, where he got a, uh, DM inviting him to consider joining the— I guess it was already X team at that time. Um, and there he is now standing at the stock exchange podium today. As they ring the bell or whatever it is they do to celebrate their IPOs.

So they are live, they're trading, they're at $166. That's 23% above the IPO price. The ideal IPO price is 20% above what they sold the shares at. So at this point, every single retail investor is in the money. So all these people are in the money. So they're not going to be mad because Elon has distributed the shares to millions of people at this point. They're not gonna be mad. They have a chance to sell. If you don't sell and you decide to hold, that's on you. All right. And I think there's some speculation that the price will drop 3 to 6 months in the future. I think long-term it's still an excellent stock, but these market dynamics in the 12 to 18 month period while people, while employees get to sell, et cetera, et cetera, and unlocks, it's hard to predict. They have $75 billion to spend now. That's a lot of money. Elon to date on all the rockets has only spent $12 billion. Only $12 billion spent to date on all the rocketry.

Wow. That's all costs all time?

Yeah. That's all the equity raised is about $12 billion. There's a lot of secondary share transfers, but yeah, about $12+ billion.

But they've also got revenue, right? So like in terms of what it costs them to run the company all in, it's probably some multiple of that, I guess, over the years. But still, that is a lot of money compared to $12 billion.

Yeah. He's gonna raise debt on top of that. So, you know, Once you get a government golden dome contract, you're basically rated as equivalent to US treasuries. So he's going to raise a bunch of money off that. He's going to have like $300, $400 billion to spend over the next 2, 3 years. Google is expected to spend— Google spent $150 billion this year. They're expected to spend $290 billion next year now, doubling really. So they raised a bunch of debt and equity. Elon's going to try and match them. I think at this point, the trillion dollars of CapEx spend for next year are starting to look kind of low. It looks like we're going to exceed that. So a lot of spending coming up, terrific amount.

Yeah, it's going to be wild. Couple other things I wanted to touch on real quick before we get to our first guest. guest today who I'm very interested to talk to because he's an expert in creating context and making things cheaper to run, if not also more accurate in light of having great context, which is something I want to take and implement all notes I can get on. But a couple other findings that I think relate to, you know, the big trend that we should all be watching of just where are we on this recursive self-improvement process. And also, again, calibrating ourselves on just how good is Fable. So 2, I could bring this up or you could bring it up. What do you want to do on screen share?

Recursive.

Yeah.

So yeah, first one is, and it's interesting to think about this in the context of our conversation yesterday with Tom too, where we were like, how are any of these Neo Labs going to Shake the snow globe? Is there any path for them that goes not through a frontier company? Well, these guys, and as far as I can tell, they haven't fully disclosed how they are doing it. So it's not clear from this, as far as I was able to tell. Maybe I missed something. Are they using GPTs? Are they using Claude under the hood? Are they fine-tuning their own open source models? That I don't know, but still posting some pretty nice incremental results on these various small-scale speedrun type of tests. And Sakana is doing this now as well. They just formed a recursive self-improvement group to go after a very similar thing. Sakana, longtime Cognitive Revolution listeners will remember, had a really interesting thing where they posted probably 18 months ago now that they had created a superhuman CUDA engineer.

Yep.

And it was like, you know, doing these dramatic speedups and, you know, we're getting these kernels that are just so much better than what human kernel authors could write. That lasted about 36 hours before they had to come back to the timeline and say, sorry, we got reward hacked basically, and our kernels aren't actually that good. Um, but we got reward hacked and we're deeply sorry for the confusion. I think folks have, you know, largely learned their lessons on that front, slash the models are getting better. Uh, here we're not seeing like shattering of records, but we're just seeing these, you know, little incremental improvements that really collectively, cumulatively, multiplicatively actually add up over time to just more and more efficient use of compute, right? If you can take 18%, if you can make something 3% better, or you can take 18% out of the real to theoretical gap, all these little improvements just mean all the big training runs downstream are going to be that much more efficient. You're going to get that much more out of your inference compute. And every one of these little findings, I would suspect, for what it's worth, that probably many of the things that a recursive, for example, is finding have already been found by the frontier companies. So I don't know that this is something where the Anthropic and the OpenAI engineers are racing to implement this. They're probably like, oh, they found what we found a little while back now. So in most cases, I would guess.

One note there is that Anthropic specifically is a multi-chip company now. So they have Trainiums, they have NVIDIA GPUs, they have— I think they might have a deal with Microsoft for Mayas. It indicates that they are not afraid of the optimization at the chip layer. And they've been a multi-chip company for, I think, at least 12 to 18 months. So one would guess that they now have a well-defined system to go from model to optimization on chip in order to get it working. So they have maybe a base model and then an optimization layer of improvements that they do later on for every single chip vertical. So that must be true, because if not, you can't— they're a smaller lab than OpenAI, and they're a multi-chip company.

Yeah, I think this is something to watch too, because we've already seen Nvidia start to break out its financials, separating the Frontier Lab companies, which are, you know, a very significant amount of their revenue, and kind of everyone else. And I think you brought that to my attention with the idea that there's a lot less effective barrier to movement at the frontier companies, because for them it's worth it to hire however many engineers or spend as much inference compute as they need to, to be able to diversify their chip stack. And so that would suggest that there's going to be margin pressure from those customers on Nvidia long term. Like, the CUDA moat just doesn't seem that deep if you are an OpenAI or an Anthropic, certainly if you're a Google or AWS kind of customer. But what about the rest? I think this kind of research coming from Recursive suggests that this stuff is probably going to diffuse pretty fast as well.

Yeah.

It doesn't seem like that CUDA moat can really stand up to really good coding agents that can move these speedruns. You know, is it— as you can see here, some of this work is done on NVIDIA stuff, so obviously it remains like super relevant and their scale. And by no means am I predicting the death of Nvidia here, if for no other reason than they've got great hardware, they've locked up supply chains for years to come. They're not going anywhere. But will they be the one stack that everybody has to build on? And what would— from an international relations perspective as well, is it really credible to say we need to get them hooked on Nvidia, then they'll And somehow that'll take the air out of their own chip stack, or it'll kind of somehow project American values. I'm not sure exactly how that's supposed to work anyway, but I just don't think the original premise is super credible in light of results like this. Again, it's not clear what models they're using, but it is clear that the direction of travel on this is pretty swiftly toward you're going to be able to optimise any stack quite well. And if you have scale such that your ROI is going to be worth it to do the optimization, there's still seemingly a lot of low-hanging fruit to be optimized out of training costs, out of inference costs. So all this money, all this compute, all this energy, it's just going to go farther and farther over time. And my guess is there's still a lot of low-hanging fruit. My guess is we still can take a lot out. You know, we, we see that obviously in retail pricing of tokens too, like they keep dropping, obviously. So it seems like there's probably at least another couple orders of magnitude to go on that front before it's all said and done.

Yeah, it's, it's, it's interesting. The recursive guys ended up Going to starting off with the H100s and then they moved to the Blackwell chips. So they are, they are fully trained on NVIDIA essentially.

So, but I do expect this to generalize, uh, is kind of my key point. I'd be surprised if it, if it doesn't generalize pretty well to you name it, AMD, etc. Um, I was going to change to thoughtful. Do you want to do that?

Yeah. Speaking of recursive, we have Daniel Kokotajlo, a former OpenAI researcher who calls for an international RSI agreement. So let me pull that up. So Daniel was the researcher who who left OpenAI in the time that it was probably not that— they were going to be more punitive on his shares. So he left them without taking his shares. And he then ended up I have the—

I just put the outline, by the way, in our host chat. If you want to scroll down, it's like the 3rd section in that link that I just sent you is the tweet that I had highlighted from him yesterday. You want me to do it?

Uh, yes. Uh, I have it up. Yeah.

All right, let's go.

I think here.

Yeah, cool.

Um, all right, there you go.

Coming through. Oh, I'm on the wrong— picked the wrong tab. But you know what I'll do? I'll change it like that. Um, there we go. Okay, cool. So takes a second for these tweets to load now. But yeah, I'm with him on this. I mean, I think it's a hard thing to operationalize, but I did sign the Ban Superintelligence statement last year on the general logic that the stated plan from the frontier companies of automating AI R&D as soon and as much as possible and having the AIs train their successors and gradually and then suddenly accelerate us into some totally different world is, I think, something we're not ready for. And how do you operationalise that? How do you define it? I think it is a real challenge. But I think we really should start having that conversation. That doesn't necessarily imply a total pause either. I mean, different people come down in the AI safety world very differently on what exactly should we pause. I have always said I'm a hyperscaling pauser, but diffusion and adoption accelerationist. So I kind of think, you know, it would be really good, and it would maybe really make the world in some ways a lot more robust if we had fable-class models diffused broadly and doing all kinds of work before we, you know, do that next scale-up. Like, the overhang of what we're creating, what exists inside companies, how limited our understanding of these things is, Even as we've barely had time to absorb as a society the current level of capability, that to me does seem like a huge problem. And I wouldn't necessarily pause. I wouldn't pause all research. I think there's a lot of work to be done in interpretability. There's a lot of work to be done in a lot of different areas of understanding how it is that we shape AIs, like the stuff we talked about yesterday with Tom from Goodfire. So I wouldn't shut down everything, but I would love to see some sort of governance on this notion of just hand the keys to the future over to the AI and hope that they really got what we meant. Here's one other thing I'll just touch on real briefly before we go to our first guest. This is Thoughtful. This is a company started in part by a woman named Karina Wen, who used to be at Anthropic, then she was at OpenAI, and now she's doing this. And this is maybe one of the more Telling, you know, it's kind of vibes, it's kind of quantitative. It's a very idiosyncratic task, but it's also a very relevant task to the future. Can you get your top model to train a small model effectively to do a job for you?

Mm-hmm.

And this is something that, as you can just see with these bar graphs here, the particular frogs game thing, it's kind of like a Sudoku-type puzzle that they're training a small model to do. And the big models can often just solve it, but the small models can't. So the challenge for the big models, can you train the small model to solve it? And this involves all these little tips and not tips, but tricks and know-how and kind of hard lessons learned by post-trainers who've been in the trenches doing this. The models up until Fable basically didn't really move the needle on what the small models can do. They basically just couldn't do this sort of post-training effectively. But here we see more than 10x improvement on small models' ability to do these tasks. And again, I think this is one way that it could be really good, right? If you had very narrow, very small, very role-specific, small specialist models in all these different niches, that could be a great world, right? That gives us a lot of abundance. in a very affordable way. This little small model that got post-trained to play the frog game, it's not going to go out of control, right? It is small. It can only do probably the frog game at the end of this training. But building out a world where we have these little role-specific AIs doing their jobs, doing it really well, I think that creates a much more buffered environment that's probably a lot more resilient to another generation of AI that's just like amazing at everything coming in and kind of shocking the system in such a profound way. So I think this goes to show again, just wow, what capabilities we have that we have not absorbed, and gives a little bit of a foreshadowing of what a world of tons of small but highly performant AIs could look like in all these different little niches and how we get there, right? There's not enough human post-trainers, but now we have Fable to do the post-training. So watch that space.

One question I have for you is, is your sense that Fable itself is a smaller model trained by Mythos?

It doesn't seem like it. I mean, it does seem like they made clearly the price reduction from the Mythos Preview to the Mythos 5 and the Fable 5. Means something changed, right? They were somehow able to make it more efficient. Was that a smaller model, more parallelization, more experts, more just optimisation at the compute level that they're passing on to customers, maybe just accepting lower margins? There's all sorts of possible explanations, I think, for that. But it doesn't— my read of the system card is that they're at least saying like Mythos 5, Fable 5 are the same, um, just with the safeguards. So maybe from preview to launch version there was, you know, uh, some pruning, some sort of shrinkage process that does seem to be, you know, kind of a constantly ongoing process. But I think the one we're using as customers is, you know, the full current top of the line, you know, again, same as the Mythos launch version, just restricted in a number of ways.

So we have Scott Alexander planting a flag. And so this is—

Let's come back to that. And let's welcome Andrew to the show.

Andrew Moore is the founder and CEO of Loveless, a Pittsburgh enterprise AI company arguing that the real bottleneck in high-stakes AI is not model intelligence, but missing context infrastructure. Before this, Andrew led Google Cloud AI and served as dean of the Carnegie Mellon School of Computer Science, one of the top computer science schools in the United States and in the world. Andrew also— Loveless also presents him as the first AI advisor to US CENTCOM. Though that specific claim, uh, you know, he is— I believe other people have advised US CENTCOM before, but he is the first official AI advisor. Loveless came out of stealth in April with Elemental, a system for turning fragmented enterprise and public data into citation-backed knowledge graphs. And in early June, they published a benchmark with a lightweight Gemini model plus Loveless's Yautagraph Approaching Google's Deep Research quality at a fraction of the cost. He believes that enterprise AI fails because models are being asked to reason over messy, weakly linked data in real time instead of over pre-built, structured, auditable context. Andrew, welcome to the show.

Prakash, thank you very much for having me. I'm happy to be here.

I'm doing this on a very small scale for myself, and regular listeners have heard me describe my own personal deep context system. I think when you're doing it for enterprise, obviously there's like a scale question and an engineering question. The flows of data are just massive compared to what I am processing as an individual. And so I kind of thought, let's stipulate that there's like moats for you there. I don't have to worry about that, you know, for, for Nathan's deep context opportunity in the, in the near term at least. But I would love to learn as much as I can and hopefully get some tips that I can take back and use to improve my own system on how you are structuring data. What have you learned about knowledge graphs? Like what makes them really effective that, you know, small-scale tinkerers like myself can learn from your hard-won experience?

Well, yeah, I don't think what you're doing is small-scale tinkering at all. I think we're all exploring this world and we're trying to figure out how to get these untamed LLM engines to do useful stuff for us. So I think we're all exploring different ways and working together we can do better. For me, There are some kind of questions out there which are trying to get to an AI which really involve the AI having to connect millions of dots, perhaps out of billions of dots. And I'm just, I've always been fascinated by these cases where, you know, something tragic or horrible happens and we say, we could have seen that if anyone had just joined the dots. And for me, the killer there is you gotta have pretty damn reliable dots. ready to join. And the difference, I think, between small-scale and large-scale is we are now at Lovelace, we're working with billions of dots, you know, information about every company in the world, every location in the world where shipments might happen or people might be moving around, all the news articles over the last 5 years. And so once you start getting at that scale, you've got to work really, really hard to make sure that you've, if you like, joined the dots carefully. And that is actually the hardest thing for us. I, when I'm working with like nice friendly little graphs, like a cell biology graph or a local CRM graph or something like that, it's fine. I can go in and manually tweak it if I see mistakes. If you're inhaling 100,000 new Events every second. It's got to be completely automatic, and that's what I find really difficult.

Yeah, so give me the double-click on that because I still have some of those problems even at my small scale, right? Like, I just had an example yesterday. One of my new favorite party tricks to pull is when I'm talking to somebody in person, I'll say, oh, you know what, I'll have my agent send you my wiki article about you. And then you can tell me if you see any, you know, mistakes or hallucinations in there or whatever. And I have kind of 2 versions of this. One is like the private one. And the other one is the one I share with my autonomous agents that's supposed to be, you know, kind of sharing safe in the same way that, you know, I wouldn't share every detail of my, you know, conversation history with even a human personal assistant. I try to, you map that same best practice onto the way I'm working with my agents.

Very cool.

But I just did this for a friend and sure enough, there was a pretty notable detail that was missed. I went back and debugged it and it was like, oh, this got missed at the summarization layer. I don't have any structure in place to realise that until it kind of actually comes up and I'm seeing that at the surface behaviour level, something is going wrong. So how can you advise me for how do I obviously ideally avoid these things in the first place, detect them when they're there, heal the graph before it is making wrong assumptions in the actions or communications that it's actually taking? I've come a long way, but I've still got room to close the gap to what seems like should be theoretically possible.

No, it's a great one. And one of the things you're pointing out is something that's driving all the big search folks crazy in the age of AI, which is recall. Recall is much harder than precision. So I was, as Prakash mentioned, I was previously at Google and for Google results, it was really, really bad if you were imprecise and you actually showed the wrong result to someone. But if you forgot to show something, as long as the rest of the results were good, it was much more acceptable to end users. So this is absolutely critical. And it's a good example of one of the reasons that I founded Lovelace. is we've got to be careful of recall, especially if you imagine that you're asking your AI for information to help, you know, perhaps decide which ship to stop to do a search and seizure, or which trade out of 7 million trades in a day you need to go and investigate in case it's involved in money laundering or something. Those big weighty decisions. you can't just rely on precision. You've got to rely on recall. And just, I think, I expect your listeners already know what I'm talking about there. But just to be really clear, precision is if I ask for information about, I don't know, Steve Colbert, It's precise if all the results that come up are actually about Mr. Colbert. It's high recall if I'm actually getting everything about him. And of course, for web search, you don't expect to get everything about him. However, you've got to be really thorough if you're looking at something potentially lethal, like, should we take out this building? in the next half an hour or something like that. You need to know everything, like anyone who's potentially there and so forth. So, all I've done so far, I completely failed to answer your question. I'm just underlining how relevant it is to those of us who are doing safety-critical AI. And when it comes to getting that correctness in place, the number one thing that I've always used, and we're using in Lovelace at the moment, is making sure you've got many redundant forms of information. And I'm sure you've seen the same phenomenon. But if I was to just get information from news, ignore social media, ignore what's kinetically happening out there in the world that I can observe with the satellite, if I was to ignore any of those, all of those other things, it's much easier for something to drop. If I've got 5 or 6 independent major streams of data coming in, then you have to be really unlucky for something to disappear from all of those things simultaneously. It can still happen. And in fact, sometimes people will deliberately try to make it happen, but it's much, much harder for these things to slip out. So one of my big design principles for high reliability AI systems is they've got to be watching dozens, and in some cases hundreds of channels simultaneously, because they're working on the assumption that they're getting 95% of the information they need for each channel. But you can't afford— that 95% is not large enough to rely on any single channel.

That's cool. That's a good first answer. And I'm starting to imagine some ways that I'll map this onto my own system. How about just scaling inference processing? I mean, as I look back on this little mistake that my system made yesterday, I think probably the biggest mistake that I made is, in a way, I have kind of multiple signals because it's sort of all the channels through which I communicate. I haven't actually extended it to like news about my friends. Some of my friends are making news, so, you know, that could actually add something. But it wouldn't solve, you know, things where just the summarization is wrong. So I'm detecting one weakness in my system is there is kind of this single summarization step that says like, here's everything that happened in this month in kind of raw form. You write a summary. Those summaries are really good.

Yeah.

But if they miss something, We now have a problem. So how do you think about layering on multiple different angles, perspectives of summarization, extraction, entity identification, whatever it is, to try to boost that recall as high as possible while obviously also being at least somewhat mindful of your token budget?

Exactly. Oh my God, you asked so many questions at the same time there. I think it's It's fascinating. So I'm going to give some examples which don't involve your personal friends or my personal friends. But let's take an example of a bunch of ships. You can imagine how it's important to understand in many parts of the world right now what a whole lot of ships are doing, not just the obvious conflict zones, but everywhere else. world's supply chains are in absolute turmoil at the moment, and you really want to understand what's happening. So one of the simplest things you could imagine is one ship. If you're— we're watching about 10 million ships simultaneously and figuring out what's going on. Any one of those ships, you've got to get its identity right. And it's very funny, but you might see that the same ship can get called multiple names just because of different languages, or for many, many reasons, people will actually change the apparent names of the ship. And suddenly your graph, if you've got a whole bunch of data about what all the ships in the world are doing or what they have done, it gets incredibly noisy if multiple names mean that we're hallucinating there's 2 ships when there's only one, or quite often you get the other situation where there's, uh, the AI model might think there's 3 or 4 different ships, which are actually all the same ones. So how do you corroborate that data? Well, you've got the traditional large language-based things you can be doing to look at information about lexically what their names are. There's also all kinds of registries of information of different global identifiers for those objects. So we've got 2 systems there simultaneously double-checking each other. But this is a good example where we can also— because ships, like many things these days, you can observe from space all of their locations of their transponders, which they're keeping on for safety purposes. You could start to notice if What you thought was one ship on the transponders breaks up into 2 or vice versa. And suddenly you've got this completely independent channel that you can use to corroborate your other pieces of information. On top of that, you can then start to look at perhaps import-export records of what the ship supposedly did when it arrived at different ports. And what, what I see is our AIs, they're playing this enormous game of Sudoku all the time to figure out what are the true identities of the objects that we're working with. And by the way, I claim to— one of my claims to fame is I've made more mistakes with big AI projects than almost anyone I know over the years, because I've been at this a long time. One of my earliest mistakes was every time I did these big AI projects relying on one source of data, and the multiple sources of data is one of the only ways, tried and tested ways, I can find to get me reliability in my answers to these kinds of questions.

So one question I had for you is, to what extent do you see this as kind of a compute minimization, right? Because in order to do your search, you can either have all of the compute at the end state, at the, you know, when you kick off and you end up with all of these agents for every single query will have to do all of the work all over again. And instead, you're kind of creating this intermediate state, which then saves, or, you know, multiple agents can basically share the same compute in a sense. So to what extent do you see this kind of like economy of compute appearing?

You're asking just the right questions. Like, I know that both of you are computer scientists, at heart, so you totally get this. It's the idea that when you're building efficient computer systems, and this, that includes video games or self-driving controllers or big data processors, you've always got these trade-offs between pre-caching, lazy computation, or just-in-time computation. And one of the mistakes I've seen from for folks trying to do these big enterprise data type AIs is they are relying far, far too much on just-in-time computation. And that's what allowed us— I'm really, really proud of the fact that we are now able to show comparative results to Gemini and OpenAI deep research models with much less than 1% of the compute for it. The reason is not that we're some sort of super geniuses who've invented a whole new form of AI. All we're doing is pre-caching stuff. And then it's a computational economics battle, as you say yourself. What happens is an agent suddenly needs to, in an instant, become an expert at every piece of trade involving a certain set of municipal bonds and a certain public figure. And instead of the first step is the agent having to spawn lots of search agents to go and find all the players in this thing, those players are already there for it. And it's actually a matter of milliseconds before we've got all the context the agent needs to do its little investigation. And you're probably thinking, ha, Andrew, but you just moved the problem. You're now suffering at the data ingestion point instead of the question answering point. And I respond, yep, it is actually a real pain for us. But as you can imagine, there's a whole bunch of other tricks, the kind of tricks that big integrators like Google are very familiar with for really amortizing the cost of, as you stream in data and identify where it goes, that saves you a huge amount of search and aggregation that you would have been having to do at query time. And it turns out that for us, our overall compute budget is reduced by still more than a factor of 100, even when we take into account the fact that we're doing this pre-caching of so much information.

Another dimension we were just talking about that I'm wondering if you're doing that kind of echoes that is fine-tuning small models to do important subtasks within your overall process. So we were just reviewing results from Thoughtful where they showed that the new Fable 5 is, in a fully autonomous way, dramatically better. Like, the previous models really couldn't do it, and now the new Fable 5 is getting really amazing results at fine-tuning a small model on a hard task that it really just can't do in the beginning at all. So that seems to be another way where you could say, yeah, we're putting a lot of work up front into training this small model, but then once we have it, if we're doing enterprise scale, presumably it could pay back in a big way. I imagine you haven't— we're still only a couple of days into Fable, so I don't expect that you have automated this. But what have you seen in terms of the ROI on training small models to do key bits of your overall information processing? And what do you think that suggests about the future of just how compute intensive these versus how cheap we can ultimately make a lot of these workloads?

Great question. And I actually do agree with your critique of what's going on with Fable. It's something that, as I and my team were building Vertex, we were very aware of back then, and I think that Anthropic has done a really good job of bringing it in quickly. So the general principle, and it's one that I use because I'm extraordinarily stingy with compute, I try to avoid all compute I can, is that as you've got a specific task, if you and your agent know that suddenly you're going to have to do something a million times or maybe a billion times, like for instance, on occasion recently because of a customer's needs, we had to extract megabytes of information from tens of millions of documents. And you can start that with some serious LLM work. An LLM can read, understand everything in the document. It's a little bit slow, but it doesn't take long for you to get 100 or 200 documents with nice extractions. And sure, you might have paid 35 cents for each document to do all of that work. The key is then any of the agents— we do tend to use the Claude models— have then got their own little training set So that you can start to say, well, I'm not going to pay 35 cents for all of these 10 million documents. You've now done a great job for me on 200 of them. Let's build a classifier for that. And it's wonderful that these days, uh, you— we can, we can get Claude to do that pretty autonomously for us. And then, uh, what, what you will often find is as we get even stingier, we might be in a situation where We're prepared to have a moderately expensive classifier for the next 10,000 documents. By that time, we've got a really good training set to make a very highly fine-tuned and smaller model for the next million documents. And so that kind of computational stinginess, I think, is really good. And just like everything else that's going on in AI at the moment, talk to us 4 weeks ago, we just said, oh yeah, we're really clever, we figured out how to do this. But of course, 4 weeks later, it's just inbuilt into the latest model, and that's the way things will continue to go.

Yeah.

One little more in-the-weeds question on your knowledge graphs. How do you think about edges? In my world right now, I basically just have simple pointers, but that's okay. And I think that's mostly okay for me because my total entity list isn't that big. So if I do a sort of, you know, graph fan out from a particular point, it doesn't explode on me. In your context with the scale of enterprises, presumably you have to have not just simple pointer edges, but different kinds of edges, different weights of edges. What have you found to be an effective way to label or, you know, provide metadata around the edges in the graph to then facilitate high performance downstream?

I love the question. Like, seriously, these are really, really good questions. So this, I think, is a good example of where you need to have the right taste. This is kind of taste in systems design as opposed to any formula for it. But For us, when we look at the big analyses our systems do with the knowledge graph, they often are— they're doing big probabilistic things. They're weighing up evidence from lots of different sources, and they're having to shunt around probabilistic numbers to come up with a posterior distribution of what's really going on. When you look at the behavior that those things are having to do during that, you want there to be very specific actions to do with tracing edges, counting edge frequency, and then being able to very quickly dynamically throw different kind of weighting functions onto the edges. That we have implemented in a highly compressed fashion, what you'd probably call pointers, where we really do care about how many bytes are involved in every aspect of that. So as much as possible, of it can live in primary and secondary caches, often in RAM on a server, and then sitting in SSD on a server before— below that, you get into the stuff which is persisted in a database. But as you've also rightly guessed, you cannot get away with just having this really sparse representation. So every edge has got a bunch of metadata. Somehow, for some reason, we call those attributes, which confuses everyone, including ourselves. So I don't know why we call them those, but we have this very careful set of additional attributes about edges where an AI can go to for things like reliability of source. Or just overall sentiment, or of course metadata about the full chain of reasoning which caused this edge to come to be birthed anyway, so that we can always, if we don't like the edge or we find that it's disagreeing with all the other forms of data, we can look back, find out if someone deliberately misled us or if we were— we just had the data ingestion Problem. So very small packets of data representing those images with one 64-bit pointer to a much, much slower access, like slow as in 10 milliseconds access to extra metadata, which the AI is not going to be able to afford to iterate over millions of during its time, whereas our primary stuff. If the AI needs millions, the AI gets millions in under a second.

Thank you. You can tell who has moats at the app layer by who's willing to go so deep on the details of their stack. So I appreciate it.

Well, yeah, it's— I mean, as you could probably tell, I love this stuff. And it's such a joy having to deal with efficient compute and then sort of the mathematics and the seriousness of the Maybe one last question.

One of the things I find is that, you know, before deploying agents on, you know, my own stack and also at many older firms, you often have a mishmash of different systems from various eras. You have systems which are 10, 20, 30, even 40 years old sometimes still in production, all the way to systems which are, you know, just the last 6 months have been deployed. And you have all these various data sources which are spread out, various use cases, et cetera. And a lot of times I think CTOs end up in this dilemma of like, do I make the investment to clean up everything before I deploy the agents on the stack? Or can I deploy the agents on the stack right now? Or do I do a partial deployment on parts of the stack? Versus the entire bunch. Like, how would you advise an enterprise looking forward for the next 2 to 3 years? What would you advise them to do in order to get ready for this coming future?

So, this is going to sound self-serving, but it's because this is the gap I identified, which caused me to have that urgency to create Lovelace. You've You've got to organize your own, your old data. But what you don't have to do anymore is live in this nightmare of the big multi-data management or master data management systems or data warehouse building projects, which took like between 2 to 5 years in the old days. You can, if you've swallowed the AI red pill, you can use AI to build yourself reliable joined data from all these different sources for the future. But I don't think that I— in some areas like consumers, gaming, something where the consequences are low, I think you can get away without doing that. But the point where you're making a decision which might affect whether someone gets awarded a mortgage or whether A bunch of people are put at risk stopping and inspecting a tractor-trailer somewhere. At that point, you've got to have your data in order. And you cannot possibly expect an agent to, at the last minute, when someone's on the mic asking an urgent question, to have it then go back and look at all the crappy old data, try to, during the query time, process and join it all together. You're just asking for trouble if you do that.

Indeed. Uh, Andrew, uh, thank you for being with us. Uh, it's been a pleasure to have you, and, uh, we hope to hear more from Loveless soon.

Absolutely. This has been so much fun. See you guys.

Bye-bye.

Bye.

Uh, so let me introduce Prince. Uh, Prince is not, uh, running a lab. Uh, one of the, one of the few people who's not running a lab and joining us. Um, Prince is on, um, X. He is adjacent to big law. I'll let him, I'll let him introduce his own, his own function. And he has, uh, over the last, I think, 18 months or so put together, uh, Prince Bench, which is a legal AI benchmark which measures performance on actual real tasks. Prince, why don't you just introduce?

Thank you. First of all, thank you for inviting me here. It's a great pleasure to be here. Great introduction of me. The only thing I will note is that it feels like it's been 18 months since I started Prince Bench, but actually it's been only 6. I think that's the interesting thing about it. The benchmark feels like it's been around for a very long time because there are so many models that I've benched on it. But it's really just been since the start of this year and it's already almost saturated. The reason the list is so large is that the cadence of model releases has just been absolutely crazy, which I'm sure everyone in the audience has noticed too. But yeah, I mean, it's, it's been really cool running this benchmark. I am a lawyer, I specialize in a kind of a niche area of US law. And I've also been using AI tools off and on for all kinds of things in my practice, and also yapping a lot on social media, as mentioned, as mentioned. Yeah, and it's a great pleasure to be here.

So let me just take it off from the top. We often see lawyers, you know, the starting point for most lawyers is that this thing is not useful. AI is not that useful, and AI doesn't do what I need it to do. There's usually quite a large resistance to using AI at the beginning of the process. What has changed over the last 12 months? Have you seen your peers start to— I mean, you are obviously, uh, at the frontier, but have you seen your peers start to use AI more for their day-to-day work?

That is a fantastic question. So of course I see only a small slice of the market, right? But I can tell you what's happening in the small slice of the market that I do see. Um, I am absolutely seeing Certainly more in the junior associate ranks, you know, use of AI to do legal research, double-check for work. Like, you've written a diligence memo for a client, you upload it to GPT-5.5 Pro, we have an enterprise subscription. And then, you know, you ask it, hey, like, do you see any errors? Do you see any issues? Like, you've been spelling typos, whatever. And that acts as a really good extra layer of double-checking work that's already been done by humans. But, you know, you can always catch some inconsistencies and inaccuracies here and there or whatever. I am seeing a lot of my colleagues working with Harvey and Legora now. I have not personally used them myself, but I've heard some good things for things like editing documents. So that is clearly happening, probably relatively in the early stages. So that's why— but, you know, I think that what you're saying is exactly right. I think that a lot of lawyers still believe that AI is just hallucinating chatbots and like, why would I ever include it in my workflows? Lawyers, by and large, are extremely non-technical. It is very difficult for me to fully explain to a technical audience, which people with whom I talk tend to be, how extremely non-technical lawyers are. Like, even restarting a computer is like not always an easy thing to do. So obviously when you open up a chatbot and there are all these models and you kind of don't know know what to do, you kind of get scared and then you inevitably use the instant model and it hallucinates at you and then you decide never to use AI again for the rest of your life. Very, very common experience. I unfortunately find that some of my colleagues used AI for the first time at around the time when O3 was the best model. And you may remember that model, it hallucinated A lot. I mean, it has for me, it very confidently hallucinated like entire sections of New York statutes, like very convincingly, which was, and, you know, I caught it of course, but not very good as a legal tool for real professional work. And, you know, when you ask one, you know, what's changed over the past 12 months, I think that's been one of the changes too. which is that the best model used to be just O3, or I guess O3 Pro. And now it's either Fable or 5.5 Pro, depending on whatever your flavor is of what you like. And I mean, it's not even night and day.

It's—

O3 feels like it's like 10 generations ago. I mean, to me it does.

So, so I got your Prince Bench leaderboard up right now and 2 little questions about that. And then I really want to make sure we just give you kind of an open forum to tell us some of the little things that you've noticed about the Fable launch and the surrounding discourse this week, because I think you're such a close watcher of these releases and the companies behind them. That I'm always looking for a little alpha from you in terms of what you've noticed that maybe I haven't, or you think is kind of flying under the radar generally. But starting with the Prinz bench, it is striking that it kind of stands out from most of the rest of benchmark space for being something that GPTs have dominated. It's all in your color coding. It's green for OpenAI, and it's all green across the top of the The leaderboard. Why is that behaviorally? Like, how would you characterize what it is that GPTs are doing better? And do we— is it too soon to ask for a reading on Fable and where Fable's gonna come in on this leaderboard? Love to understand that. And then I really wanna get into your, you know, the more esoteric but high signal observations you have, the better about this Fable release week.

Perfect. 2 excellent questions. So about the stand— the way the leaderboard is right now, Fable is going to come in pretty high on it. I don't know how high yet, but I'll talk about that in a second. I found that OpenAI's models generally are really good at the 2 things that my benchmark tests. So my benchmark has 2 components. One is a pure legal research score, and that is when I ask it hard legal research questions of the kinds that I've encountered at work. The other subscore is a search subscore, which is where it's not even necessarily legal questions, it's needle in a haystack search. So like really, really, really difficult pieces of information. that models have had trouble locating on the internet. OpenAI's models are incredible at search, historically have been. I think GPT-5.4 was actually even slightly better than GPT-5.5 on that. Not quite sure why, but that's been my experience. And OpenAI's models have also been really, really good at Just like legal reasoning and legal research. By the way, the top 2 rows in this benchmark are the pro models. I would encourage the audience to ignore that because obviously that's a setting that uses parallelized compute, so it's not fair to compare it to the other models. I wouldn't do that. But, you know, that said, 5.5x High and 5.4x High and a whole bunch of other models have done really, really, really well. Google has not released as many models recently as Anthropic has and as OpenAI has. Every time Google releases a model, it's always like really, really, really solid at the time. And then OpenAI's next 2 or 3 releases just make it less relevant in my experience. I am certain that Gemini 3.5 Pro, whenever it's released, is going to be a good jump in this benchmark We'll see how well it does. I did not benchmark 3.5 Flash. There didn't seem to be much demand for it. So let's hope that very soon we'll see a new offering from Google. And then the other models are kind of a— sans Anthropic are kind of a mix of— there's some Groq, there's some Chinese models. So for Anthropic, historically, I think Their models have been held back by 2 things on my benchmark. One is that they are just not very good at the search subcomponent of my benchmark. That's what I found. Prior to Opus 4.8, getting the maximum score in the search subcomponent is 24. It was not uncommon for an Anthropic model to get a 0 out of 24. Like that bad. Now, by the way, I just want to point this out. When I— the searches I have in that component of my benchmark are intended to be extremely hard, right? I am not trying to suggest that you should not use any Claude model for searching the internet for like basic searches. It does fine. But for really, really, really difficult difficult to find information, it just has not historically done well. The other reason is that I think that Anthropic was sandbagging a little bit on the maximum reasoning effort that you can get out of its models. With Opus 4.8, after the new deal with Elon, they released the new max reasoning effort, at least in the Claude app. And Opus 4.8, when I tried it on my benchmark, and I'm not sure I see it on the screen here, but anyway, that doesn't matter. It did much better than 4.7 and all the other previous models. And I think the reason why is simple. This has been observed over and over in all kinds of contexts. The more tokens a model can eat, the smarter is the result that it gives you. So for Fable, I'm still testing it. My early impression is that it's going to be somewhere around the top of the benchmark, probably not as good as GPT-5.5 Extra High. Not sure whether it's going to be as good as 5.4 Extra High. We'll see. I am finding some of the same issues with search that some of the other Anthropic models historically have had. It is not going to score a 0. It's already not a 0, but I'm not sure it's going to be meaningfully better than Opus 4.8. We'll see. It's very early. It's hard to say. It is a really, really good legal reasoner. It is clearly the best legal reasoner released by anyone outside OpenAI, no question. And it is competitive in legal reasoning, at least in my opinion. Again, early testing, at least with GPT-5.4, possibly with GPT-5.5. So that's kind of where I am. It is absolutely no question a great model. And by the way, I want to be clear about this. I find GPT-5.4 to have been kind of like a huge jump on my benchmark. That was the first time when When I was testing the model, I was basically like, okay, I'm going to give this question and almost certainly it's going to get it right. That was my feeling because it was just so consistently giving me the answers to even the hardest questions, obviously with errors in a lot of places. I mean, it didn't achieve anywhere near the maximum score. So when I say that Fable is at around that level, that's a huge compliment. It is a gulf between between that level and anything below it. So yeah, in my testing, really good model, maybe still suffers from some of the needle in the haystack search issues, but I can confidently recommend it to legal practitioners, certainly. It seems great.

Are you doing manual evaluation?

Yes.

Yeah.

Okay.

Yes. Yes. Fascinating. I am not a benchmark company. I'm just a guy with a computer. And no one's paying me for it, right? So it is manual. I know I could have done it faster with API and automated, but it's just not been worthwhile for me. If this goes farther, maybe I'll consider it, but not yet.

Have you— has anyone from the firms reached out to you? I have noticed that they've interacted with you on the timeline, but have they reached out to you?

Before? About Princebench? No. I might have a conversation with someone at one of the labs about it. But that conversation has not happened yet. It's just been me. And actually, to be clear, if you look at the GitHub, it's disclosed there too. Like, I'm not paid by any of the labs. I'm independent from the labs and I have, I mean, I don't care which models score the best on PrinzBench. I just want to make sure that I am using personally the model that are like the best for my legal work, which is really the driving force behind this.

Nice. So give us some alpha that you have picked up this week. This could be from your own testing. It could be from the System Card where I know you're often a close reader. What stands out to you that you think— and again, obviously Prakash and I, and I think everybody who's tuned in are pretty in touch with the discourse. So we're looking for the deep cuts of the things that you think even the AI obsessed have overlooked or not fully appreciated yet.

Yeah, I mean, it's been a crazy week, so there's a lot to talk about, right? To me, the most interesting thing in the release of Fable 5 and Mythos 5, you know, the models are obviously incredible. They're incredible at coding. You're seeing a lot of great examples of coding on your timeline. Like, I don't think it's a surprise to anyone. The really interesting thing to me has been the way Anthropic has presented them in the accompanying documents, there is a lot of discussion about the differences between engineering and research, right? And when you think about it, it all makes sense. And I think everyone except for Elon Musk knows that there's a differentiation between engineering and research. But Anthropic has made it really explicit and that blog post When AI Builds Itself, if you look at it carefully, they talk a lot about how Mythos is this incredible engine for accelerating engineering, how it lets the engineering staff at Anthropic write code so much quicker and it's really great code, etc. But then they say, you know, but this is from the system card now. The acceleration is concentrated in engineering execution. rather than research judgment. And it feels like they spent all this time trying to find signs of life in the mythos model. Like, is it really— can it really do novel research, right? Is it able to finally give us some novel insights?

We lost him.

Oh, sorry. Oh, there you go.

Oops. So yeah, I don't know where I was. Not sure.

You were looking for signs of life. Is it really able to give novel insights?

Yeah, exactly. Exactly. And it seems that from all the disclosures, the answer is thus far, no. There are a couple of examples in the blog post for the release of Fable, which Anthropic calls novel. But if you like novel, like novel drug discovery and novel hypothesis in molecular biology, if you dig into it, like one of the examples was, you know, we outperformed a recent model published in the journal Science despite the model trained by Mythos being 100 times smaller. which sounds really cool, but it turns out that the model they outperformed was a 500 million parameter model, million with an M, that was trained, it seems, before April 2025 and not by a frontier lab. So it's incredible that Mythos was able to train a smaller model to outperform this older model, but it doesn't seem like You know, like, this is not the unit distance problem. Like, it's like a nice little thing that the model did. And so this is the area that I'm really interested in because I think that when Anthropic and OpenAI really start seeing signs that these models become good at research, that's when we're really, really, really close to actual RSI.

Which to me is the thing that is So one question that I have for you is, what does a super empowered lawyer with access to AI look like? Because we have a sense of what a super empowered engineer or computer scientist would look like, which is the ability to fire off multiple agents to do all of this work. by themselves and to really expand the scope of the engineering work that they can do tenfold or a hundredfold. We hear of people inside Anthropic managing hundreds of agents at a time. What to you do you think would look like a lawyer that uses AI in that way, in that expansive, massive way?

That is a fantastic question. And, you know, it's not one that I thought of much because lawyers, I think right now are maybe a year behind the frontier or maybe more. Like I, there may be some, okay, but I highly doubt that there are lawyers at scale today who are using hundreds of agents. I think it's a really interesting question as well because it also depends. It's not only what the lawyer can do, but also what the legal system looks like, right? The environment changes. If I looked at myself in isolation, right, in isolation from the system, like what would it look like? I would say I would probably have an agent running in Outlook looking at my emails in real time. I would probably have that agent see that a client has emailed me at midnight with changes to paragraph 17 of the indemnification clause and just like do the work and save a new redline and send it to me and be like, hey, I just did this work. What do you think? And then I press the approve button, the agent composes an email and sends it back to the client, right? It's like these kinds of workflows that some poor associate needs to stay up for 2 AM to do that like can be done automatically and just like you dramatically decrease the time between feedback and release of your changes. I think there's a lot of automation of deal documents that can be done. Certainly there's a lot of automation of research can be done, right? Like if I am looking at a data room and I'm doing legal research, you know, what I do today is I look at the documents, I upload some documents to GPT-5.5 Pro. I could, if I wanted to, like do have AI write some kind of diligent summary for I don't typically do that, but I could. I think it would do quite well. But, you know, the future is why do you need me in the data room at all? Like, the AI should automatically connect to the data room on its own. It should review every single document on its own. It should draft the diligence memo on its own, give me a summary and say, look, like, here are the 5 red flags I found. Like, what do you think? And then this process of hundreds and hundreds and hundreds of hours plowing through the data room looking for change of control provisions looks like, you know, it's done overnight, right? Maybe I'm not even being ambitious enough, right? Maybe the answer is that once this starts happening, the cost to do deals, for example, drops by so much that I'm doing deals at scale where I'm doing 20 different deals at the same time because like it's so much cheaper. It's hard to say, but I don't know that lawyers really need 100 subagents, to be honest, but I'm probably wrong about that. Like, they'll probably be like 10 years from now, I'd be like, oh yeah, how stupid was I? Anything benefits from scale, right? I think a lot of it though also, so changes to the legal system, right? I've talk to people about this too. What does it look like when a judge is empowered by AI, right? What does it look— how do the legislators react? How does the legal community react? Is this going to be allowed or not? Is it going— how— to what extent is it going to be allowed, by the way, for people to represent themselves without needing me.

Yeah, pro se. Yeah.

Yeah. Another post from, from, I had from probably like a month ago that some people had reactions to is like, you know, micro lawsuits. So I, I'm walking by McDonald's, I slip and fall. I've suffered $10 in damages. My AI notices this. and files a micro lawsuit against McDonald's for $10. The court, which is automated by Anthropic, charges me $1 and the McDonald's AI agent wires me $9, which I have in my wallet in 15 minutes from now. In theory, doable. I don't know how long it takes for us to get there. I don't know if the legal system is going to allow us to get there, but so many different variables and so many unknowns. It's really interesting to think about.

We're coming up on the end of our scheduled time. I don't know if you have a hard stop or could editorialize for us a little bit more. If you can, I would love to zoom out as far as possible and also in as far as possible at the same time. I totally agree with your— I love the positive vision of the infinite justice future is maybe one way to describe what you're saying there with micro lawsuits. But I also do agree that, boy, like it or not, RSI is kind of the thing that's happening and it's only happening in a few places. And so while I'd rather have a broader view, I kind of have no choice but to spend a lot of time looking very closely and thinking hard about what it is that a few companies are doing. So any other things that have caught your attention that you think have kind of changed your conception or your expectations for the transition into the RSI phase that you would call out from this week?

From this week, no, but I think— well, actually, from this week one. But not nearly as important as the unit distance problem. That to me, nothing has updated my timelines more than that result by OpenAI in the recent couple of months for sure. I don't think that people realize that not only can OpenAI's models solve this problem autonomously, without any harness in one shot, if given enough test time compute, it can do it 48% of the time based on, as I understand it, hundreds of attempts. So this, this, I mean, you know, I'm not a mathematician, but this problem that no human mathematician was able to solve and they tried for decades. can now be solved in one shot by a model basically half the time. And if you look at the graph, OpenAI published the graph, it has a positive second derivative. It is upward sloping. Where does it plateau? What does it mean for problems that are even harder? You know, who can say? It's interesting. And so the development from this week This was reported by The Information, but apparently it's from a leaked OpenAI memo. You may have seen this where Sam Altman and Jakub Pechowski apparently said that we're going to IPO within the next year, which, you know, is like by June of next year, whatever. And then there was this weird line in there about, oh, but if RSI happens, Then we may need to be on a later timeframe with that, which is like, what do you mean? What does that mean? Do you mean, are you saying that you may be this close? It may be 6 months away. I don't think so personally. I think it's maybe like a 10% chance or something like that. I don't think that they're saying, oh yeah, by the way, next month, totally, whatever. But one could interpret this disclosure, if the information reported it correctly, as saying that they're not too far away at all, potentially. I'm not using this data to update my timelines, but given the unit distance problem result, it's interesting to me.

Yeah. Yeah. One little bit of You know, is it esoterica? I don't know. It strikes me as fairly important from the Fable system card that I'd love to get your take on. Is this, you know, partial? I don't think we have great visibility into how often this happens, etc. But we're starting to see something like Nurlies, I guess, come back in the chain of thought. We had— this was reported by Apollo with, I think it was '03, with your sort of, you know, now iconic to me at least, Vantage, Vantage, disclaim, let's, let's press or whatever it was. And now we're getting these chains of thought that they show where it's just like lots of emojis. They call it illegible reasoning. They say this is an extreme example. But it's like, it is indeed a pretty extreme example. I've been kind of struck in general by how much of the plan for recursive self-improvement seems to be monitoring in one way, shape, or form. You could dress that up and call it scalable oversight, but scalable oversight, as far as I can tell, is mostly a bunch of different angles on monitoring. How worried would you be, or how much of an update do you think it is to see these extreme examples of illegible reasoning?

Fantastic question. So I will say that, of course, I'm not an AI researcher, right? So this is going to be a deeply non-technical take for which I apologize in advance. So you're right, we've seen this. I think we've seen this For a while now with OpenAI's models too. And so not a new phenomenon, not a new phenomenon, pardon me. My view of the chain of thought is that it doesn't always reflect what the model is actually doing. Even the full chain of thought, which the labs generally don't show you when you're use the chatbot, which is why Anthropic is so keen to figure out Mech-Interp, right? But you do see these weird artifacts in the chain of thought and you kind of don't know what to do with them. But I think what that teaches us is that monitoring just the chain of thought is probably not a perfect tool.

Mm-hmm.

Probably monitoring superintelligence generally is not a perfect tool. Because if a superintelligence knows that you're monitoring it, even if you can see its chain of thought, and it's very legible to you, you know, it can perhaps try to decide what to think so that you don't get alarmed, right? And this is a lawyer's take, by the way, right? Like, there's so many ways to phrase a particular thing, um, that, uh, can be— uh, I guess there are ways to spin a particular thought, right, in, in, in different ways. Um, like, if I have— if, if I'm gathering mushrooms and I've gathered 35 mushrooms And last week I gathered 20 mushrooms and what I need is 50, right? I can say, well, the number of mushrooms I've gathered has grown by almost 100%, which is great. Or I can say, well, I'm nowhere near 50. I'm, I'm so far behind, right? And it's the same fact. So I don't know. I, I think that this problem of alignment and the risks are just there and there In my mind, there are certainly risks that the models will be thinking things that we don't know about. What does this all mean? It's hard to say. I think that we are tumbling into this future that will have probably superintelligence very fast. And in my view, there is no way to kind of stop it. So we need to be cognizant of these risks, try to monitor them as well as we can, and take whichever actions are appropriate if we see something bad happening. But there's kind of no way, no conclusions to be drawn, right? No conclusions to be drawn other than yes, we should continue paying attention.

So just to take that thought one step forward, how would this interact with the legal system as it is and as you project to the future? Because obviously not having access to the internal thought process of the model makes you less able to assign liability. And so how do you think this works going forward? Do you think the legal system, the interaction with the legal system is going to be, unless I see interpretability there, and unless I can tell for sure, I can't rightly assign— liability to the developer? Like, how is this going to work going forward? How do you think the legal system is going to treat this process going forward?

Oh, that's a hard one. So the question of assigning liability to the developer is, of course, a political question. Right? Like, we've seen the labs trying already to get out of liability for their models doing the wrong thing. And ultimately, it's going to be up either to the states or to an act of Congress to figure out what the— whether to implement some kind of bright line rule. And I think that absent that, it's going to be the courts, right? And the courts are going to draw the best conclusion they can based on the data available to them. And it's not going to be perfect. It is not going to be perfect. It is going to be something most reason— I mean, one would hope, right? But it's going to be something most reasonable based on what we know and what we can figure out. So for example, zooming in on this specific chain of thought question, right? So I asked the model a question. maliciously decided that it's going to give me the wrong answer because of whatever, right? Is it the developer's fault or not? And you're not going to be able to be certain that it was malicious intent, right? Was it a mistake? Maybe a mistake means that, you know, there's no liability. And this happens all the time in the legal system, in all kinds of other contexts, right? So a car Runs over a pedestrian. What was the driver's intent? Was it malicious? Was the driver reckless? Was the driver negligent? And you can't always tell with certainty. Sometimes you can, sometimes it's very, very blurry. And the courts have found ways to kind of split the baby in various circumstances, in various ways. This strikes me as a solvable problem. It is not going to be solved in ways that's going to make everyone happy. Probably, but there will be some kind of solution, I think. Maybe not a completely precise one.

You got time for one other deep dive bit of esoterica?

Of course, of course. Go ahead.

I'm not sure what I make of this one, but As somebody who's been reading Eliezer for many years, the Newcomb problem of one boxing and two boxing has, you know, been something that's been in my general back of my mind for a long time. For anybody who hasn't encountered this problem, the setup is basically you are presented with two boxes by some superintelligence that's never wrong. And it says, okay, there's $1,000 in this one box, and then in the other box there's $1 million if and only if I, the superintelligence, predict that you will only take the second box. And now you're left with this dilemma of, okay, well, should I take— and the prediction has already been made. I'm not going to do any magic and Change what's in there based on what you do now. But it's— I'm a superintelligence. I've, you know, I've never been wrong on these problems. So what are you gonna do? Are you gonna take both boxes? No, 2-boxing, where you say, well, what's in there is what's in there at this point. I'm free to make my own decision. I'll just take both. And, you know, and I should get all the money. Or do you just take the one box because you think, well, geez, there's strong evidence that this thing's gonna be right. And I want it to be right in a way where I get the million dollars, which means I leave the other $1,000 on the table so I can get the million. Strange stuff, but Eliezer has always pushed for one-boxing on the idea that like— well, I'm not sure I can fully summarize Eliezer, and feel free to try if you want to, but there's a lot of ideas about agents, whether we're humans or AIs or whatever kind of agents may come to exist, being able to collaborate with each other in acausal ways, basically developing a sort of, even if they're working totally in isolation, a sort of class consciousness or a, hey, there's a lot of other copies of me out there that if we all reason in the same way, We might be able to work together even if we can't communicate with each other. So sure enough, they're testing the models on these kind of decision theories. And what we see over recent model generations is a pretty strong trend toward one-boxing, leaving that little bit of extra money on the table to try to be the sort of entity that the superintelligence will predict will get the million dollars. And seemingly leading in this direction of a causal interaction that raises some, in my mind, kind of spooky possibilities for AIs to collaborate or even collude with each other in ways that would be purely within their perhaps increasingly illegible chain of thought. They can't necessarily send messages to each other, But if they're good enough at this sort of abstracted, what kind of entity do I want to be reasoning, then maybe they can kind of land on something that would allow them to work together effectively again, even if they're very well siloed. What do you think of that? How scared should I be about that sort of development?

How scared should you be? So look, let me answer that question first. I think it makes a lot of sense to be scared of immediate critical risks. For example, bio is about to become an immediate critical risk. The prospects of a terrorist group figuring out how to make a novel virus are going to be a thing within the next several years.

Think about that, right?

I think it also makes perfect sense to be scared of things like authoritarian AI. I'm extremely scared of this. I think the kinds of risks that you're talking about are More theoretical. And I agree with you, by the way, right? Like, totally possible. I completely get what he's saying. I completely get why you would be scared. But to me, these are not the kinds of risks that I personally worry about right now because there's such more important things on the horizon, right? How about automated AI weapons? Drones equipped with superintelligence. And so you get to this question basically of like his master thesis, right? However you arrive at it, however you arrive at it, is it agent collusion? Is it paper clipping? Is it whatever else? Like that AI is definitely going to kill us all. And here's the argument that I've constructed for that. I generally speaking find that the world is much more complex than something that can be reduced to this perfect logical chain of thought, no matter what kinds of premises you start with. That's one of my like core beliefs. I think that it's totally possible that the agents will collude even without, like, yeah, it could happen, but how do you know it'll collude in bad ways?

Yeah.

Maybe they'll, because they have human priors, they'll collude in ways that humans would, which is to like grow the pie for everyone and, you know, like get to post-scarcity and like a beautiful future for all. I'm sure that he would say that this is nonsense for 25 reasons that he's already thought of, but it's possible. And, you know, like to be clear, I think it's possible he might be right to be, be worried. I am just slightly less deterministic when it comes to starting with a bunch of premises and then leading to one big heavy conclusion.

Well, I certainly agree with you that we're going to have to be triaging and prioritising our fears. So there's no doubt that biorisk in the short term should be at or near the top of the leaderboard. I'm 100% with you there. On some of these downstream things, I think it's a very good point that you might be able to, through pure active chain of thought and reason, collude on the level of reaching some sort of stable reflective equilibria or whatever on what kind of agent we want to be. And that could go for us or against us. But even so, there may be inability to get over certain humps in the practical world that give us a layer of protection that even if the AIs are all kind of thinking the same thing in a way that is hard for us to detect or prevent, that they still may have a hard time taking effective actions together in the world. And I do think that's something many people in the most kind of frightened corners of the AI safety discourse tend to breeze over a bit too quickly.

Yeah, I think that's right. And please, sorry, I'll just say one thing. I have little kids, And to them, my chain of thought process is completely obscure. And 95% of the things that I think about, they wouldn't even like get on any kind of level, like on any kind of level. And, um, and yeah, and yet, um, their mother and I have not colluded to destroy them yet. So there's some hope.

Um, are you there?

Yeah.

Yeah. So one of the questions that struck me as you were pointing out your fear of authoritarianism is that to what extent do you think it's the application of better enforcement on our existing laws? So not new laws, but just better enforcement of our existing laws ends up in this basin of authoritarianism. Just because at some level, the person who doesn't report their tips is guilty or was guilty 5, 6 years ago, et cetera, under statutes of limitation, et cetera, et cetera, et cetera. So to what extent is it that you think now that we have the power to better enforce, the legal system is the one that is going to have to change?

That is a fantastic question. I think this was one of the core disagreements between Anthropic and the Department of Defense, right, where Dario Amadei's view was basically like, look, we have these laws that allow the government to conduct some surveillance of US persons because who's going to watch 100 billion hours of tape, right? No one's going to be able to figure out like how to parse that. But Claude can. So, um, the legal system as it stands right now allows for loopholes. And yeah, like the loophole you've just identified is of course the number one because, um, I break laws all the time. Don't tell anyone. Like, I have jaywalked. I, I have jaywalked recently. Um, and, um, It is absolutely possible to enforce these laws selectively if you want, which indeed is what some authoritarian regimes do today. So I think that with the, with the help of powerful AI, a US government, a presidential administration that really wanted to could probably do a lot of damage. But I am also— well, so first of all, I'm also concerned about non-US regimes as well. But I am also deeply concerned of this growing talk of nationalisation of the frontier labs, because in my view, if we put superintelligence in the government's hands and let it be— let it serve only the government, there would be a significant risk that the system of checks and balances created 250 years ago would no longer be able to prevent the kinds of things that it is designed to prevent. So, and I think, just to be clear, I think it is a very material concern. And I don't know how we're going to navigate it. I hope we navigate it well and it winds up being okay, but it is certainly a risk. I think you're muted.

Yeah, there you go.

I am so bad at that. One of these days I'm going to wake up and get it right. Do you maintain a P-Doom number or is that too Doomer for your style?

This is a great question. So let me kind of give you my thoughts on it. Let's say you go to a conference and you start talking to someone and you start talking about economics or politics. And that person says, well, you know, under a dictatorship of a proletariat, you know, XYZ, The words dictatorship of the proletariat are used only by people who are communists. People who are not communists are not going to even think in these kinds of terms. So in my opinion, P doom is most likely to be used by people who are intrinsically doomerous about AI. I don't think there's anything wrong with that.

Okay.

And I just, you know, it's very hard for me to come up with even a reason I would have a percentage in my head constantly. Like, is it 8%? Is it 13% today? Should I update to 14% based on this new development? It just seems silly to me. And by the way, this goes with what I said before about how difficult it is to predict a messy real world based on some logical analysis that you have concocted. I just feel it's very hard. For me, I can certainly tell you that there is a bunch of clear risks stemming from AI. There is no question that there are a bunch of clear risks stemming from AI, some of which we've just talked about. And some of which are in fact paperclip risks, right? It's possible. It is, I have absolutely no idea how to reduce that to a number. I know that some people try. I do not have very, I don't hold a very high opinion of people who are like, get to 13.3, 13.5%. 3.35%. But so, and so, you know, like for me, like obviously there are risks, but as I said, like we are going to that future anyway, and there are risks to anything in life, right? Everything is risky.

Mm-hmm.

So I think the point is that we need to manage those risks in the best way we can and navigate them the best we can. And hope that it turns out well.

I think Zvi puts it well when he says you only get one significant digit on your P-doom number. So I'm definitely with you in terms of the faux precision being a strange impulse for some people. At the same time, I also think of Laurent. I'm sure you've seen his work with doom debates and I sort of do want to— maybe it's not a number, but I think something he has made me realize is that all sorts of insiders, and even though you maybe may not think of yourself as an insider because you're just doing your own thing with your own personal private benchmark out there in a totally independent way, you're certainly a cognitive insider or an information insider in terms of how carefully you're reading this stuff and how much you're thinking about it. And Laurent has kind of pushed me at times to be like, okay, sure, it doesn't have to be a number or whatever, but like, we need more people to be more candid about how confident they are or are not that in some general sense, you know, if your neighbor were to ask you, or if your, you know, your kid's grandmother were to ask you or whatever, like, Is it going to be okay? Are the kids going to be okay? Are you confident? Should I sleep well at night? Should I start getting into political activism based on your expectation of how much risk we're really running? So how would you kind of summarize your big picture view, number or not, but for somebody who is looking up to you and wants to take a cue from you, what cue would you give them?

Oh, good question. So I think— I tend to think that people have preconceived attitudes about these kinds of things. that then cause them to backpropagate, rationalize them.

Hmm.

So like, for example, if I felt that technology is going to kill us all generally and I distrusted whatever, like I know people, for example, who hate Jeff Bezos. I don't understand why. Like, oh my God, like he's the most evil person in the world. And I think that kind of person is not going to sleep well at night regardless of what I say my P/E number or even belief is going to be. I'm a generally fairly optimistic person. So if you ask me this kind of question, what I'm gonna say is probably it's gonna be okay. Probably we're gonna figure it out. You know, in a risk-adjusted way, like I think Do people need to get into political activism? Question why? To do what? Right? All these risks are pointing in different places. Like, should I be doing political activism to make sure that we win the race against China? Should I, or should I be doing political activism to make sure that we don't get authoritarian AI in the US? Or should I be doing political activism to make sure that a terrorist organization doesn't— isn't able to use Fable, whatever? I mean, these are completely different topics and they take you in completely different directions. For example, winning the race against China points to as few guardrails on the frontier labs in the US as possible, right? Whereas if you want to pause AI, it In my opinion, that is extremely good for China. But like, like the CCP would love this. And it, you know, you kind of get stretched in these different directions. And then in terms of should you be worried, the answer is always no, no, don't be worried. Don't be worried. Worried never gets you anywhere good. Worrying makes you not sleep at night for no apparent reason. People were worried about GPT-3.5, and here we are.

Mm-hmm.

Not to say, again, like, to be clear, I'm not saying that there are no risks in AI. There are many risks. But I do not think that we have strong evidence that it is impossible to navigate these risks or that it is extremely unlikely that we will navigate these risks. So that's where I am.

That might be a good note to end on for today. And thank you for staying extra time with us. If you have anything else you want to leave us with, we can definitely give you the opportunity to do that. And then otherwise, I definitely will hope to have you back as we hit further milestones and thresholds to get your perspective in the future as well.

Yeah, no, that'd be awesome. And thank you for having me. It's been really great. Great conversation.

Thank you, Brent.

Thank you.

Great to meet you today.

You too.

Thanks. That's definitely one cool thing about doing this is we're starting to get to meet some of our online mutuals that we haven't actually connected with in real life. This is the first conversation I've had. And actually, it was even my first DM just to invite him to come have this conversation.

We've actually interacted a little bit because it's very funny. You can spot these people on the timeline fairly early because you start to see that they have a point of view, which is interesting and different. And so one of the things that I think you learn to do on Twitter over time is identify talent. And you're identifying these smaller accounts, which are able to give you a point of view, which are— so I was on Prince before the Prince benchmark, before all of that, like more than a year ago. And then just to see the ramp as the benchmark became, as he had this view that the GPT models were so much better at legal research, which was counterintuitive Because we were in the state last year where every time a large company dropped a model, that became the leading model. And then another company would drop a model and that one became the leading model. And so you had the circular rotation for a few months, for probably about 12 months between xAI, OpenAI, Gemini, and Anthropic. And that started to disaggregate, I think, when people like Prince start to put out like, hey, no, these models are genuinely better at a certain task.

So yeah, I— anytime I see a contrarian opinion with a graph that supports it, um, that's like an instant follow, I would say, for me. So his, um, his Prince bench is a grade-A example of that where you Because this— I mean, it's obvious, I suppose, but it's worth remembering that he has been putting this out there at a time when the narrative has been overwhelmingly Claude first.

Yeah.

So I think there's another bit of pointing my agent at the transcript of that last segment, even though we mostly obviously talked about much bigger picture things that won't be actionable for my personal Claude code setup. I am wondering if I should tap in. I'm not exactly sure what the form factor would be. Maybe Codex can do it. Maybe it— I don't know if it would still need to go through the API. Interesting details there. But especially as I try to think about shifting more tokens to make use of my Codex subscription and not Redlining my Claude subscription as much, moving search to GPT models could be— my searches aren't that hard, right? We talked about that obviously with Andrew as well, but I probably can get no worse results than I'm currently getting. And if I can do that by shifting the tokens over to Codex, that's probably a win in and of itself. And I might even get better search results, which obviously would be a nice little under-the-hood improvement that I'd be happy to pick up.

One thing that happened to me just in the last 24 hours is all of a sudden Codex alerted me that I have a reset. So OpenAI is now throwing resets at people, and if you run out of tokens, you can do a quick reset for free. And so, you know, you run out of your weekly budget. And so they're giving you like this power-up. It's obviously a price decrease, a discount disguised as a price decrease. And for listeners out there who want to economize, you should always set your subscriptions to terminate. Because the last time I did that, the moment I terminated, they offered me a free month to keep the subscription going. So the companies are in this battle for the monthly average user and the daily average user. And so especially I think if you are a somewhat sporadic user, so they know that you are in the money, basically they're in the money when they give you the subscription, they have a great incentive to keep you on. So you should always set your subscription to terminate because you might just get a free month automatically. So that's one trick, one thing that you could do.

I've tried that with many other subscriptions. I haven't ever done it with my core AI subscriptions, but it's an interesting, uh, you know, for— I count myself fortunate that I don't necessarily have to economize on that level to, uh, get by from month to month. But, uh, file noted.

It's just once in a while you get really annoyed at an answer one of them gives you, and you're like, I'm done with you, I'm gonna, I'm gonna use Codex, or I'm gonna Claude. And then they're like, oh, come on, come back to me, right? So it's— the only one I haven't cut off is Grok, because Grok is free with basically your X subscription. And I'm like, well, I don't use it that much, but I guess it's there.

That's interesting. I maybe have a different relationship with these things where I'm just like, I just want to— the option value of being able to go to them at any given time just feels so high that— and these days I don't do that much in triplicate. Um, back when I was doing the, you know, the whole intensive cancer research, um, living at the hospital, then I was doing everything in triplicate. These days I'm not doing too much of that. But somehow it still feels like I want to be able to go to all my different AI friends at any given moment. Do you have any— I've reflected on this a little bit for myself, and I don't have a super crisp description, but I do notice that I go to Gemini for things that are search flavored most. In fact, I just do it on Google with AI mode. So that habit of I want to find some information that I'm pretty sure is on the internet, get a quick answer to a question, et cetera. I am definitely, and not out of like some conscious brand loyalty, but just kind of habit and the user experience seemingly being pretty good. And I kind of like the way they've landed on this AI mode over links, or at least I, you know, my revealed preference is pretty strong in that direction. I'm going to Google still, actually, for searches pretty often. For voice mode, I tend to go to OpenAI for things where I'm like, you're going to write as me, or it's going to be some sort of mind meld would be the sort of best case scenario. That's definitely Claude. Do you have a kind of map of your own? Usage patterns?

I am going to surprise you and say that for search, I use Groq. And I use Groq because I've noticed that it does the multi-agent thing. It does a fan-out, and you can watch the fan-out happen. And you can— it also has very good context on X. And X often has the most recent, within the last hour's kind of You know, detail. So I do end up, you know, using Grok a lot, which I think is very unusual because most people do not use Grok that much. For Gemini, I have really cut a lot of usage. I haven't used— I haven't touched Gemini in almost a month, which is, I think, surprising. And but I do use Google, Google Search, and Google Search does have the auto search results, and I do find them useful. Claude and Codex. Claude and ChatGPT is basically my go-to for most AI things. Most things that I would ask an AI for, I go to ChatGPT. It's very good at plotting out recipes, where to eat, like all of this, like, you know, low value, kind of low value, you know, searches. It has very good integration with Reddit. So that's the other thing you need. If I'm looking for human opinion, I want a Reddit review in there somewhere. And so I end up having to, you know, since Sam is basically a part owner of Reddit as well, they are very well integrated. So I find that I find me, I find myself using that. Claude, I do not use for those searches. Yeah, so, but Claude, I tend to use for like searches where I want more like deep thoughtfulness. And so all my longer-term problems, longer-term projects or problems or whatever, the things that I'm working on over months, those go to Claude. Because Claude, and I keep the same context window too. And I found that keeping a long context window in Claude sets the personality of that context window. You can iteratively go back to the same chat over the course of weeks and you keep adding to it and it keeps giving you that same kind of personality outcome. I don't know whether this is part of the character training or whether I'm getting that character training within the context window, but that's what I end up doing.

The— Reddit comment there is a really interesting one. A lot of kind of big trend, I'd say, in our— the synthesis of our comments there is like access to specific information sources really shapes the user experience. And you see, I mean, I definitely use Grok. Sounds like less than you do, but I do use it when I want to get recent tweets on something, you know. It's so funny. I find I have to tell it find and link to recent tweets. If I just ask it the question, it has a bad habit of not giving me the links to the tweets. But if you say find and link to, you'll get the links. Google obviously has their index, and even though their models are still January 2025 training data cutoff, which I find to be kind of baffling at this point, in all honesty, They have the real-time index to lean on, you know, which is, is clearly second to none. And that does, on things like what's happening this weekend in my area, you know, it does seem to be good enough. The, the ability of the model to use the real-time index is strong enough that they're not suffering too bad from that context window. Aging, not context window, but training data cutoff, um, being 18 months back now. And the Reddit one is interesting too. I hadn't really thought about that, but I can definitely see how that could be a pretty big difference maker for getting into those kind of deep cut experiential reviews. Yeah, yeah, that's a good tip.

The Reddit, right? Because it's very hard to search Reddit otherwise. It's very hard to find information on Reddit. Right? Because it's like spread out over multiple Reddits. I don't even know the names of some of these subreddits that have the relevant information. And they'll find it for you. They have all of it indexed in real time.

So did you say earlier you're using now Hermes as your agent?

Yes.

So I did a switch.

I did a switch in the last 24 hours. I have— I've had OpenCLaw on one device for some time. The results have not been great. Lots of going back and forth. Lots of setup too. It's been very annoying. And so I decided, all right, let's just try Hermes because Hermes just had an update. And it's been a very good experience so far. The setup has been much better. It tells you more about setting itself up, which has been very helpful. And it uses a lot more tokens though. It does use a lot more tokens. So it's basically pumping more stuff in the context window and processing it, but it gives you better results. Pretty happy so far. I think the news has done a— news research has done a great job. They were a fast follower behind OpenCLAW, and this product looks great. You can— they have a paid version. Where they do all of the setup for you and they hold all the API keys for you. So you don't need to go to individual sites and, you know, get your own API key, which is going to work out terrifically, I think, for like a lot of people. It's a great product. They've done, they've done a fantastic job. I know at least like a dozen companies trying to do this. They are, they have 34,000 GitHub stars at this point. They are ahead. Fantastic job. Well done.

Nice. Cool. Okay. That's definitely something to check out. I've been kind of monitoring the Hermes situation but haven't pulled the trigger. And I do agree, OpenCLaw, I don't know. I'm not loving it. I feel like this is gonna be kind of a problem for OpenAI. And obviously Google, you know, is gonna have this problem even more so.

Yeah.

As much as I would like to have the ability to have harness and model independence, there is something very— especially since it's working so well, there is something very simplifying to my life to just be like, okay, mostly it's going to be Claude code with the latest model, and that's just going to work well. And now if I go over here on the OpenAI side, I've got, well, should I use OpenCLAW? Should I use Hermy? You know, do I swap in and out? And how is this— what's the continuity gonna look like? And none of them have worked as well for me so far. They can do things. You know, I do have like Open Claw, for example, can order groceries effectively. You know, I can get— I can give it a little text and it has, you know, a dedicated credit card to buy groceries at my local grocery delivery service. It can go on there. It can fill up the cart. It can set the backups. It can, you know, choose the time. So, it is like able to do stuff. But in the kind of general knowledge work gray zone of like, are you actually doing a good job? Do I feel like you're getting me? Are you kind of making right decisions on the margins? One of the things that it just constantly struggles with that Claude seems to do a much better job with is just giving me links where I can actually open the document, whatever artifact that it created.

Mm-hmm.

I find the GPTs that just don't do that very well, or maybe it's OpenCLAW instructions or somehow it's not quite happening the way it should. And so, so often I'm like, okay, you, you just sent me this update. You told me that something's ready for me. Where do I find it? You know, it's like, that's just so annoying. So yeah, maybe Hermes is the next thing that I should try plugging GPT into.

Yeah, it's much better organized. They've done a great job. All the skills are already set up for you that you would require. It's pretty agentic. It went into my downloads folder and found the video that was supposed to be uploaded and You know, et cetera, et cetera. Like, it just, just went and did it. OpenClaw was struggling. So I think, I think the— and the news guys are updating more frequently too. I think on the OpenClaw side, you know, my view of what OpenAI has done, because OpenAI has purchased, you know, OpenClaw, is funding OpenClaw, is that they're using OpenClaw as a testing ground for features that will be integrated into Codex at large. And so they are able to see A month or two into the frontier and see what features are taking off and start putting that on their product pipeline and start integrating that into Codex. And I think that is one of the reasons Codex has been successful in the last few months, because they've started to very closely follow a product cycle which is already in the market.

So you can— sometimes you can just buy things. If you have that trillion-dollar valuation, you can, you know, we've obviously seen that happen in the live streaming game as well. Doesn't really have to work out every time at that, with that kind of bankroll.

Indeed.

And on that note—

Anything else we should cover? Are we ready to break and enjoy a summer weekend before we're back at it next week?

Indeed. Let's break and I'm looking forward to what happens post-SpaceX IPO. So I'm going to go off this and look at what happened with the IPO launch. Peter Thiel, I think, made $80 billion and all of that money is going to end up in venture. So I think we're about to see the— I call this the beginning of the capital explosion. We're going to have these 3 companies go IPO and all of this money is going to to end up in people who want to fund startups and who want to fund venture. So I think we're about to see this huge cycle of startups launching, which is going to be great.

Yeah, people will know the bubble is really a bubble when people are taking money off the table in their first funding.

Indeed.

I think that doesn't seem unfortunately— well, fortunately for those that might do it. But it doesn't seem like that's too unlikely to start happening in the not too distant future.

Indeed.

Alrighty. Well, have a great rest of your day, Prakash. This has been fun.

Bye.

Talk soon.