Dean Ball, on Joining OpenAI: New Power Centers, Frontier AI Policy, & Main Character Energy

Hello, and welcome back to The Cognitive Revolution. My guest today is Dean Ball, who, as you probably already know, recently announced that he'll soon be joining OpenAI to build and lead a new team called Strategic Futures, with a mandate to help OpenAI's senior leaders shape frontier AI policy. The obvious importance of that position as we enter the era of recursive self-improvement, with OpenAI's own public timeline calling for an AI research intern just 3 months from now, and a full-fledged autonomous AI researcher in March 2028, just 21 months from now, makes this one of the most self-recommending episodes that we have ever done. And because Dean was so generous with his time, we were able to cover a ton of ground. In the first section, we get Dean's reflections on America's AI Action Plan a year after its release, and his perspective on the current state of the Anthropic Supply Chain Risk Designation, And the moves that state governments have made in the absence of preemption. We also get his understanding of the Chinese government's decision to restrict the purchase of American chips, what he thinks is happening behind the scenes right now with respect to the ongoing Fable ban, and how much he personally misses Fable as a user. We then turn to his reasons for joining Frontier Labs now. He explains that Frontier Labs are a fundamentally new kind of powerful actor. Which demand new policy paradigms, and also critically, that the information they contain about the present and future of AI development is so differentiated that he feels he simply won't be able to do his best work without access. He also describes how he understands his duty to OpenAI's mission of ensuring that AI benefits all humanity, how his team will relate to OpenAI's existing government affairs team, and while he emphasizes that he'll soon be learning much more from the researchers doing the work, He shares his baseline perspective on what recursive self-improvement is likely to mean, and the still neglected but critically important question of how to govern the internal deployments of the latest and greatest models. Along the way, we also get his takes on the corrigibility versus character debate, what happened between OpenAI and Alex Boris, the equity sharing talk we've recently heard from both Bernie and Trump, whether the AI industry is already too big to fail and thus implicitly government-backed, What sources of leverage AI companies have vis-à-vis the US government. How he's thinking about working personally with Sam Altman and the role that individual personalities will play in shaping the future. What success looks like for him in this role and what red lines could theoretically cause him to quit. And finally, how he intends to use AI and also to refrain from using it in his work and writing going forward. Somehow, amidst so many important issues, what stands out most to me is Dean's sense, which I share, that we are entering a main character energy period of history, a time in which individual human agency achieves maximum leverage, in a few key places at least, before perhaps the machines ultimately surpass us. It's a stark reality that forces each of us to ask, what sacrifices and compromises we are willing to make to help shape the future. Dean, whose first son is not even a year old, will clearly be making some sacrifices as he reenters the arena. I have no doubt he will be working harder than ever. But importantly, he did not compromise his intellectual independence to take this role. To my pleasant surprise, even as an OpenAI employee, he will retain the freedom to write publicly about AI policy. A freedom that extends even to this podcast, which definitely contains a few notably candid moments, but which OpenAI did not ask to review and has not seen prior to our publication. And so, without further ado, as he prepares to join OpenAI for a career-defining and to some degree a world-shaping role, I hope you enjoy this frank conversation with the great and increasingly powerful Dean W. Ball. The Cognitive Revolution is brought to you by Mercury, the fintech that more than 300,000 ambitious companies and individuals trust to run their finances. I've wired AI into nearly every corner of my life, my email, my messages, my calendar. I even gave Mercury virtual cards to my agents with low limits and category and merchant restrictions for their autonomous use. But still, my AI's access to my financial data has remained limited. With a normal bank, I might export a bunch of statements and have my assistant process them for me. But for real-time, up-to-date information, and certainly for taking any action, trying to get your agent to use the bank via the browser is just too hard, too slow, and too error-prone to be worth it. And that's why Mercury's new conversational interface, Command, is such a big deal. It's built directly into Mercury, which means you get natural language access to your finances without exposing anything outside of your bank account. No exports, no spreadsheets, no pasting your transactions into third-party tools. I really think a lot of people are going to prefer it this way. And it can already help you take actions too, with everything bound by the permissions and approval policies that you've already set up in your account. I am genuinely impressed to see this level of AI integration in banking in 2026. And so I invite you to join me in the future. Visit mercury.com to learn more and apply online in minutes. Mercury is a fintech company, not an FDIC-insured bank. Banking services provided through Choice Financial Group and Column NA, members FDIC. Thank you to Mercury for supporting The Cognitive Revolution.

And now, on with the show. Dean W. Ball, author of The Hyperdimensional Substack, welcome back to The Cognitive Revolution.

Thank you so much for having me back, Nathan. It's great to be here.

I'm really excited for this conversation. We've got some big news in your life to cover, and, you know, you've really been in the eye of the storm, uh, over the last year and a half. And so I, I have just so many questions of, Everything that you've participated in, your thoughts on where we are today. I'm going to try to go Tyler Cowen style on you and just fire a bunch of questions and mostly just want to hear from you on, on so many different topics. Are you ready for a podcast sprint?

I'm ready.

All right. Let's start off with your time in the White House and a little look back on America's AI Action Plan. It was super well received at the time. How would you critique it now? Is there anything that you feel like you would change or do differently looking back at a conceptual or policy level?

What you have to remember about the action plan is that like, you know, I, I don't feel like there's a lot, there's a ton about the world in terms of how AI has developed that has surprised me. Like basically we're still in the basic, you know, I kind of figured like, yeah, models with scary cyber capabilities late '25, early '26, and, uh, you know, probably bio soon after and kind of like that was sort of, I mean, I had publicly predicted a lot of this stuff, so I, I like, you know, Basically, we're living in the world that I figured. I think coding agents maybe surprised me in terms of popular uptake on the positive side. I didn't expect that. But the thing is, is that DC wasn't living in that world when the action plan was written. So the action plan is like this weird example. It's like this strange hermeneutics where you're writing a document now and your audience is the present-day audience, but your audience is also, you're trying to model the same people, but like in a slightly different near future where they're like 30% more AGI-pilled than they are today, and then 50%. And then you hope that they go back and look at the document and are like, oh wait, like I now read this in a totally different way now that I'm thinking about it in this way. Right? So, but I think you could definitely criticize that as being like an act slightly too much of like, you know, 5-dimensional chess or whatever. It wasn't intended to be, it's just sort of like, I think it was the nature of the task. I think the one thing I might critique is simply that it probably would've been good to be a little bit more explicit about Like we are really talking about generalist agents that can do all kinds of stuff, and like sort of trying to explain not just not just what that will mean for America, but also I mean a big part of what the action plan is all about is like okay this is happening right this is happening and the government's not leading it, and the government needs to figure out how to ride. with the current of the river and use this to maximize, in my view, American primacy, American geopolitical power, things like this, while also understanding that the best way to do that is to be positive sum and to try to grow the world economy and to try to bring other people in the world in on this. And I feel like the action plan doesn't really stitch that together all that well and probably reads a little bit more like, 3 dozen separate sort of thematic objectives than it does one cohesive thing that is unified by a common strategy or a common vision. And I think probably if I, if you'd given me 2 more months, I would've focused on that. The other thing I will say is the things we left on the cutting room floor that I, I feel, you know, bummed about. Uh, um, you know, one of which is I was really passionate about adoption in particular sectors. And trying to talk about, okay, what are the— doing case studies in really specific industries and saying, all right, what are the barriers here that the federal government can do something about? Hospitals, right? Hospital record keeping, for example. Are there very specific in-the-weeds things that the Department of Health and Human Services could do? Or Veterans Affairs. This is another one. Veterans Affairs, right? Amazing.

Huge.

One of the largest single-payer healthcare systems in the world. We don't think about America as having single-payer healthcare.

Well, we do.

The VA. Huge amounts of data, huge amounts of direct medical care being provisioned by government employees. This is the kind of thing where experimentation with AI in healthcare could have been just enormously valuable. We just didn't have time. So I would criticize that as we could have been more specific there.

So then you handed off the baton saying at the time that you're more of an ideas guy and it'd be left to somebody else who you think would hopefully be better at implementation and, you know, running all these things, um, through the actual process of government.

Mm-hmm.

How would you say that is going right now? It seems like we clearly have, like, the buildout is happening. And, you know, even in my home state, not too far away, uh, in Michigan, there's a gigawatt data center just broke ground despite some local NIMBY-style objections. So that seems like it's happening. We hear about the military trying to use AI. Obviously, that's, you know, kind of contested in terms of what they should be doing. We also don't, as a public, I don't think fully know what they're doing. And then there's all these other things. Your writing in the meantime has sounded the alarm in a pretty severe way around just the health of the republic. And it seems like your kind of faith in government's ability to ride the wave as one might hope is not super high right now. So how would you say it's going in terms of follow-through and, and where, what is the kind of core of that? pessimism for you?

I think if you looked at the individual items in the action plan, and some of this is hard because some of the things on the action plan, some of the implementation ended up being done, as they say in government, on the high side, which means in classified environments. So especially some of the stuff about military adoption, and there's some national security things. For example, one thing that I feel like is underrated is like, not to say that this specific thing has been implemented, but just as an example of the kind of thing I'm talking about, It's like, um, there is a part of the action plan that obliquely references the notion of the military commandeering all the data centers in the country, uh, in the event of a national crisis where we needed to stitch them together to do dot, dot, dot something. Uh, like that's like, you know, in there and like you wouldn't, it sounds like a very like Leopold Aschenbrenner idea. It was described in sufficiently mundane language that I feel like it didn't jump off the page at people. But so there's a bunch of things like that where the implementation, to the extent it's happening, is not happening in public settings. But I think if you were to look overall and if you had a clearance and you could really see everything, I think you would see that we're probably, if you just think of it as a to-do list, it's probably 30 to 40% done, which is pretty good for a year, right? We're about 11 months out from when the action plan came out. So like, that's pretty good. And a lot of the major things, you know, I think across all of the pillars we've seen significant advances that the administration has made on the energy side of things. Not that this was directly in the action plan, but this administration is doing really amazing stuff on nuclear. And then stuff that was more directly part downstream of the action plan, there are major changes that are in process right now that should be announced in the coming days really from FERC, the federal— Uh, uh, energy regulator that deal with the, uh, process for connecting very large industrial, uh, electricity users to the grid and accelerating that process. Um, so there's a lot of things like that are just really meaty, substantive things that are proceeding apace. I would also say, yeah, military adoption of AI has impressed me to the upside. And generally speaking, also military's direct involvement in like really industrial policy and boosting startups, US manufacturing, which is talked about in the action plan, startups that are doing innovative things with physical autonomy and stuff, that is all doing great. Um, and I think another main thing is the action plan talks a lot about adoption more broadly, and I think AI adoption in America is actually going pretty well, all things considered. Now, of course, that's all the nice stuff, I would say. I think more critically, it would be, it would be hard to say that the administration has carried itself According to what I think of as the spirit of the action plan. It's not my job to say what the spirit of the action plan is. Ultimately, it's their administration, right? For example, a big pillar, a big principle of the action plan was the notion of exporting American AI and getting it adopted all across the world. Seems hard to imagine how that's consistent with global export controls on frontier models imposed with a 90-minute notice on the entire, on all non-US persons. That's the kind of thing that when we were out in the world and, you know, my, my life after government, there are definitely a lot of international trips I do where I am engaged in quasi-diplomatic work on behalf of the United States as a private citizen, but as someone who's trying to explain what we were thinking with things like the export promotion work and like our whole strategy there. And the biggest concern you hear from people abroad, especially in Europe, is I just worry that you Americans are going to turn off the models at some point if you get mad at us. And when I was in government, we were trying to assuage this concern. When I left government, I spent time in India at the AI Action Summit earlier this year. Many places, many quasi-diplomatic engagements where I said, no, don't worry, we don't want to do that. We want an ecosystem, blah, blah, blah, blah, blah. And then of course, like, The administration goes and does it and basically confirms the biggest fears of a lot of people internationally. And that doesn't help, right? That certainly doesn't help. I think actually that relates to one other thing, which again, I don't know if it would've been possible, but one thing that I feel as though the action plan was relatively silent on was like the issue of AI governance, right? That's a big part of what I worked on before and after and like, It's not really that referenced. And I think my reasoning for it at the time would've been like, number one, tough Overton window within the admin at the time. And number two, a lot of that in my view is ideally legislative and the action plan was not supposed to talk about, it was supposed to be just things the executive branch could do, not new laws. Um, so, but I think that there could have been more explicit material in the action plan. About, hey, okay, at some point things will get scary and what should you do? Here's how not to panic. There could have been more of that because I think that we are seeing, we are seeing. It's funny though, because in the end, I do think there's just a distinction, right? There are like, there's this huge array of civil service bureaucrats, just like full-time career civil servants. And then there's low to mid-level political staff. And they all read the action plan and are like implementing it, I would say, like in quite a good way. Then of course there's like the very high-level people who don't necessarily read every strategy document that the administration comes out with. And like, they're fundamentally very reactive. And so this mythos stuff happens and it's, oh my God, we gotta do something about this. And they're not thinking about what would the action plan tell me to do? That's not at all what like a cabinet secretary is thinking. But Something that is amusing to me is that I am, I think we are watching the administration reinvent some of the ideas in the action plan, or like the senior level people in the administration reinvent some of the ideas in the action plan around like the use of the CASEI, building technical competence in the government, all that kind of stuff, third-party evaluations, all this. I think we're seeing them reinvent those things from first principles. So I'm optimistic still, but, but yeah, no, I definitely think That there have been substantial ways in which the administration has departed, ironically, in both the not taking the risk seriously enough direction, and then also in overcorrecting and taking them, not taking them too seriously, but like reacting in ways that don't actually deal with the risks. And so, you know, look, in the end, I'm inclined to give grace to people and say what I hope that we're in right now is a, is a high neuroplasticity. Phase of policymaking, and so I really I think that we might be in a very different world in three months. But certainly, yeah, there have been ways in which like if you were to look at the headlines, you'd be like, "Ah, it doesn't seem very consistent with the action plan at all," and I can't deny that.

Hey, we'll continue our interview in a moment after a word from our sponsors.

I've got a text.

The summer might be almost over, but a new bombshell has entered the villa. Meet the voices of Deepgram Flux TTS.

I'm Drew, low-key casual.

I'm Alexis, an upbeat morning person. I'm Haley, always. Flux TTS has some real personalities that are ready to speak. You can interrupt, pause, and keep talking without losing the plot. Fancy a chat?

Come try all the voices free through September 12th at deepgram.com/keeptalking. Terms apply.

So obviously one of the biggest moments, and I don't want to rehash all the politics of this because you've commented on it extensively, but one of the biggest sort of freakout moments was when the Department of War declared Anthropic to be a supply chain risk. And I'm struck by the fact that, like, it seems like we're memory-holing that kind of, you know, we're in this zone now where, as far as I understand, many areas of the government were involved in using, testing Mythos. Like, there, there's like a lot of Anthropic at the government still, is what I kind of understand the situation to be. You correct me if I'm wrong on that. How should we understand like where that whole supply chain thing is today? Are we just gonna all pretend it never happened or what?

The broad way I would describe the American presidency really since Obama's second term is, um, you know, Obama famously said, you know, he was faced with an intransigent Congress, to put it, to put it generously for him, faced with a Congress that wouldn't do any, wouldn't pass any laws. And so he said, fine, I have a— He said famously, I have a pen and a phone. And so what he meant was, I'm going to do executive actions. I'm going to push the limits of executive actions. That began a kind of autocatalytic process in which every president pushes the bounds of executive authority in various ways that get tested in the courts. And so what will happen very frequently is that you'll see a headline where it's like, Ex-president did this thing that is unprecedented with executive power and it's being litigated. And then it goes through a very long litigation process and most people lose track of it. The walks don't. It's still going on, right? Anthropic had their trial in front of the DC Circuit last month, and I think we're expecting a ruling in that trial at some point soon. And then after that, Anthropic, if they lose, Anthropic will appeal. That I'm quite certain. And if it's interesting, if the government, the Trump administration is actually very, very, very savvy about when to appeal things and when not to, when to, when to, you know, they're actually, they're pretty good reading between the lines of, okay, yeah, that one was probably illegal. We're not going to appeal that all the way up to the Supreme Court because we're going to lose there. They're actually pretty good at that. So, but I think they, I think the Trump administration actually thinks they will win this case. I think so. The litigation is ongoing is my point. And if it keeps, it would not surprise me if in the summer of 2027, we have a Supreme Court ruling of some— by the summer of 2027, there's a Supreme Court ruling of some sort on this issue. Even if that ruling is them denying to hear the case, which is the most common thing the Supreme Court does is deny to hear the case. That's also going on in terms of government use. It seems as though basically after Mythos, I think the message that was that Anthropic received was, you, the supply chain risk thing applies to your contracts with the Department of War proper. And so I think within the Department of War, they really are winding down Anthropic and have been considerably. And I think they probably wouldn't surprise me if they're 100% off of Anthropic by the end of the year or a year from now or something. But Throughout the rest of the government, I think the message was like the supply chain risk thing doesn't apply to any other government agencies. So if there's other government agencies that want to use Anthropic, that's fine. There's also technically speaking the National Security Agency is a part of the Department of War, but it seems as though not only does the National Security Agency have a contract with Anthropic, but if reporting is to be believed, Anthropic's red lines around Domestic mass surveillance and autonomous lethal weapons were honored by the National Security Agency. And that's, to me, that's a good— I think Americans are not that good at tolerating ambiguity. Um, but like a little bit of ambiguity is, is, or maybe even a healthy amount of it is just, I think, an intrinsic part of this whole process. So yeah, it's still happening. The supply chain risk thing, it's still being litigated. I think Anthropic contracts are indeed being canceled at the Department of War. And at the same time, I think other use of Anthropic in the government is going fine.

The US government contains multitudes.

Yes.

You were also, I mean, you were extremely critical of that whole situation. I think you were less critical, but still somewhat critical of the recent EO and the move, as I understand it, to take certain AI testing characterization responsibilities away from Casey, which I think now has kind of an uncertain future. I don't know what you think its future will be, and move those responsibilities to the NSA where they may or may not already be classified information. What's going on there? Like, why, why are we taking things away? Is this as simple as it was like a Biden project and so we don't like it, or is there— more going on there than that meets the eye? And why are you concerned about the sort of testing models you don't know exist, you know, against standards that can't be disclosed? Like, how do you think that turns into a problem for the public?

Yeah. So the reason I'm like, I'm critical of the administration here in terms of like where they're going, I was critical of the cyber executive order when it was signed, um, because I anticipated exactly this, Where, you know, let me just, to level set for the, for your listeners for just a moment, what does the Cyber Executive Order do? One part of it is we're gonna create a variety of procedures by which we're gonna patch vulnerabilities in critical software systems. Great. Okay. Thumbs up. I don't think anyone can object to that. I think we can argue about how, how useful those programs are gonna be and like how good the implementation will be, but we'll see. So that's one thing. Second thing is, um, It created a voluntary pre-deployment program, a testing program 30 days before release, whose details were to be classified, primarily classified, and primarily run by the intelligence community. And within that, probably practically speaking, the NSA is the primary since the NSA is the highest in terms of cyber expertise. And they really are quite excellent, by the way. Like, I think the reason my— I think this is setting up a potentially very bad future where access to frontier models is gated. Um, it's all kept secret and the public doesn't really know what's happening at the frontier. The government is making a bunch of decisions that maybe the public doesn't even know whether or not to restrict certain capabilities. about what to do with those capabilities. And it feels to me like if you believe that what's happening right now is one of the most important things ever to happen in the history of technology, that not only just do I think just intrinsically as an American, my intuit— my gut, a strong gut instinct is the public has a right to know. About what's going on. But number 2, I actually just think that it's not some trade-off. It's a better world where the public knows. It's a better world where the public knows and to the extent possible, It's a better world where the public can access frontier capabilities. First of all, I think government monopolization of frontier AI is potentially how we get very scary outcomes from a civil liberties perspective. And that's not, by the way, that is a criticism I would make regardless of who the president was. If I didn't know who the president was, if you didn't know the party, didn't know the name, if you erased that information from my brain, but I had everything else in my brain, And you told me about this, I would say I'm concerned about that. That's one thing. Another thing is that dealing with a society and a civilization is a kind of information processing system, right? It's like there's like a bunch of, all the humans in the country are at parallel compute, right? And we're all trying to process what's going on here. And while there's a lot of, there's a lot of things we don't have good answers to yet. I actually do think that the community of like AI policy people has made reasonably good progress since 2023 in terms of, okay, how practically should we be dealing with models of the Mythos-level capability? And when things are public and there's a legislative process, for example, that is informed by lots and lots of robust public input, that stuff really, that's the whole design of our system, right? That stuff can make its way in. And we can take advantage of this kind of parallel compute. When you centralize everything and make it private, it's much more brittle. And it's like a bunch of people who, as I said earlier, often don't have a, they have a million things on their plate 'cause they're high-level government officials. They often don't have a lot of context for AI, and they're just improvising, making decisions in an improvised fashion. And I think that leads to subpar decision-making. I think it leads to wasting time. Because you, it's, I feel like what we're watching right now is the administration speedrunning the sort of mentality there. They remind me very much of where DC was in the spring of 2023 when it was like ChatGPT had just come out and, oh, we're gonna regulate the hell outta this and this is really dangerous. And then things softened and maybe they softened a little too much and I just, but they still, they, Ultimately, we met in the middle. There was a change in the vibes that started in 2024. And I just think that right now we are going to, like, by putting this all in the echo chamber of the administration with a pretty small number of voices contributing to things, information and insight to things, I just feel like we're not leveraging the best that we have. And so that I think is the biggest problem. And That's why I am critical and I am very worried about the direction of policy. But unlike the supply chain risk thing, where the supply chain risk thing was like, I think just totally an own goal, just totally unforced error. Like, why did you pick that fight? You didn't need to pick that fight. You could have fixed this in a million different ways. But that, that did, you could have dealt, even if you take the government's concerns in that issue seriously, you could have dealt with that in a thousand different ways. This is more— yeah, I'm not surprised. I'm not surprised things are going about this way because you are building this thing, you're improvising an AI governance regime from scratch, and it's being built by 20 people, 15 of whom don't have a ton of context for AI. I'm not surprised it's working this way, and I'm just pointing out the meta problem of, yeah, we need to bring this out into the public. We need to have Congress involved. We can't just— this is not gonna work. Um, and so there's no point in being highly adversarial and critical there just because I'm not— what do you want 'em to do? What do you want these people to do? I don't blame them, but I do ultimately think that we need to make things more public.

On the note of parallel processing, how about the role of the states, our laboratories of democracy? A lot of the proposals that I understand that you have favored or even championed, including mandatory safety plan publication, certain other transparency measures, whistleblower protections, and even a sort of Fathom-style public-private regulatory, you know, hybrid structure have all happened in different states to, I think, a remarkable degree in a pretty short period of time. How bullish are you on the states?

So there have been really meaningful wins for the whole, the general notion of private governance that I started to work on really post-SB 1047 veto in late 2024. And of course, other— it's not just me. A lot of other people have worked on this stuff. Auditing, independent verification organizations, which would be like third-party private bodies that would like evaluate a lot of the thing right now. Like with the government, they're concerned about the jailbreak, this potential jailbreak of Mythos or Fable. And it would be great if there were expert bodies who had looked into this and really probed and certified like, hey, yeah, like there are jailbreaks cuz there are always jailbreaks, but our calculated risk is that. These jailbreaks are not severe enough to rise to the level and we can certify Anthropic as conforming to safety best practices or whatever, right? It's like the kind of thing that I think there have been substantial wins. There have been substantial wins in the sense that 2 of the 3 big AI companies have published multiple documents that are favorable to this general notion, Anthropic and OpenAI. A bill to mandate auditing in frontier AI companies passed in Illinois earlier this year. And also the state, states of Connecticut and actually the common— no, Connecticut's a state. State of Connecticut and Commonwealth of Virginia both passed earlier this year that are specifically authorizing either studies or pilot programs for independent verification organizations. And there's also a bill pending in Ohio, by the way, which would be the most robust implementations of independent verification organizations yet. Full momentum in that regard, more than I would've guessed a year ago. In that sense, I think the states as laboratories of democracy idea is working fine. It's also worth noting with respect to the frontier AI safety laws that have passed that the states have taken great effort. There's been a bill in the transparency bill in California was SB 53, in New York it was RAISE, in Illinois it was SB 315. And the language, Illinois adds an auditing requirement, but the transparency language across those 3 states is remarkably similar. Remarkably similar. And, and so I'm like quite, I'm happy about that, right? That's not creating a patchwork. Those are the, that's the states converging on a common framework that we'll see if it works. We'll see how well it works, but it's states converging on a common framework, which they do from time to time. There's a lot of other areas of AI that are not so much paid attention to on Twitter that are not so much, that don't get as much of the mindshare. But where I think the story for the states is less rosy, things like consumer protection, algorithmic pricing, the number of media, like synthetic media and deepfake laws that exist in this country now is just crazy, right? There's hundreds of them now. And the net effect of that is probably to create a fairly confusing political environment. I think also one thing that's really problematic that we're starting to see bubble up from the states are basically occupational licensing protections. States say, including Illinois has done this, saying we are gonna define mental health services as exclusively something that can be provided by humans. And if you, if a chatbot so much as asks you how you're doing, it is engaging in, or if you say to the chatbot, I'm sad, can you please help me? The chatbot is technically engaging in mental health services and That's illegal. And we'll see states often vary in terms of how rigorously they enforce laws like this, but it's not good to have that kind of stuff on the books. And so I think strangely enough, the area that gets the most attention, the frontier AI safety stuff, where a lot of the pro-preemption crowd also focuses their energy, that focuses their attention, that's actually the area where the laws are like best sculpted. It's like the laws are like very well sculpted. They often have the support of the AI industry. They're often designed specifically to avoid the patchwork complaint, which is a legitimate one. And for some reason, up heretofore at least, most of the people that are like super pro-preemption focus on these laws. And it's not only that, but these laws are dealing with really urgent problems like cyber and bio and stuff that are like clearly not fake anymore. We can't We're not having that argument anymore. Clearly a real thing. So, and then they're like not focusing on all these other areas where the states actually are creating patchworks and are creating complex compliance things that might specifically be really complicated for startups to deal with. And I think it, I think the state issue is mixed in that way, but also the people who are, who including me, who support a federal law are not helping themselves. Because they're talking about this issue, I think, in largely the wrong way.

One big surprise, I guess I'm— first question, high level, what have been the biggest surprises for you? I'll offer my biggest surprise. You can react to that and share your own. Biggest surprise for me is the administration eased the export controls on chips, and then, which wasn't shocking unto itself, But then the real shock is China doesn't want to buy them. So we had all this debate around to what degree should we, you know, are we being bellicose in, or at least I was, you know, asking that question in trying to do these export controls. Finally, they get eased and China's like, no thanks, we're going to just build our own industry and you guys can keep the chips. What's going on there? And any other surprises rise to that level for you?

So that particular development doesn't surprise me that much because China's system is very— it's actually, realistically, we are becoming more like China, um, in this regard. But one thing about China is that when China announces a new policy, it's like you need considerable expertise. And there are people inside the U.S. government who specialize in this and they do not share their opinions publicly. But it's actually very hard to get good analysis on things like this in the public discourse. One of the things I miss about government. But it's like what the policy says, and then there's what they're actually going to do, which are importantly different things. So the policy, I think it's a matter of national pride for China that we are building our own AI chip ecosystem and we don't need the Americans anymore. And they, I think they like sending that message to the world. I think they like sending that message to their own people. And there's some aspect of that. Then there is, but then there's what actually happens because while that's going on, China's system has lobbying too. And I guarantee you that DeepSeek and Alibaba and Zifu and all these other people, I guarantee you that they're like begging the Chinese, they're begging Beijing for access to American chips. And so the policy planners in Beijing are factoring that in and probably there's some amount That's being sold. I think that I think we now know that there are some chips being sold, but yeah, no, they're going to keep it. They're going to restrict it, and that might be a big own goal on their part. Well, I guess we'll see. But but yeah, no, I think in terms of U.S.-China, nothing has especially surprised me. I guess I would say I anticipated that by now that in terms of China, I anticipated that by now the Chinese state would have woken up to the catastrophic risk issues. And that they would have pushed, they would have started pushing back on the open source strategy. Um, I called it for, I said that by Q1 of this year, in, in 2025, I predicted that by the end of Q1 of 2026, DeepSeek's top model would not be open source. And that prediction was wrong. Um, I still think it's going to happen at some point, but we're not there yet. The Chinese state seems more concerned about labor issues than they seem concerned about catastrophic risk. So they, they're like less cat risk pilled than I would've guessed if you had asked me a year ago. On the technical side, nothing has really surprised me. I, one of my hobbies is I pay attention to these weird subgroups of very normal people who use coding agents. So there's this community of homeschooling moms of who love Claude Code and OpenCLAW and stuff, and they're like using it to do all kinds. It's, I love that. I wouldn't really. have guessed the coding agents becoming so popular. And then the only other, on the technical thing, the only other thing that really surprised me, I did not expect the world sim stuff in the, on the physical world, the sort of the models where you can stimulate a 3D interactive environment, basically like creating a first-person open-world video game, but for arbitrary settings in the real world. I did not anticipate that. That I, cuz the way those models worked for a really long time was they were very dreamlike where you could create the world, but it was like the neural network is creating it in real time. And so if you turn around and look at something and then turn away and then you go back and look at that thing again, it's totally different, right? It, it doesn't have, it doesn't have permanence. And then like one day, It just worked. It just— oh wow, we just have permanence now and it just works robustly. That substantially increased my timelines for robotics, um, working because it's very clear that you, you'll be able to make synthetic data pipelines. You'll be able to use human data as baseline. You make people wear an Apple Vision Pro, use that as the baseline. You can bootstrap there to synthetic data in all sorts of world sim settings. And then probably just sprinkle on a little bit of data with really high fidelity, like people wearing gloves with electrodes in them to sense muscle movements and whatnot. And it's very clear that, okay, like dexterous manipulation. The second I saw the WorldSim, it was a Google, it was a DeepMind model like last summer. And the second I saw that, I was like, what? Okay, dexterous manipulation in robots is gonna be solved in 8 months. That was, and that caused me to change my research agenda a little bit after I left government and accelerate some of the work that I was thinking about for robotics.

Yeah, it's all happening. Yeah. Um, Jim Fan from NVIDIA's little 20-minute keynote recently about the parallels between the path that he expects robotics to take and the path that LLMs have taken, I think is—

Oh, the Sequoia talk.

It's must-see TV for sure. And it definitely has me convinced as well.

Yeah.

Also shout out to Jessie Janais from the, uh, homeschooling moms, uh, contingent. I, I love the stuff she's doing and try to borrow from it. as much as I can as well. Last night for the Mexico-Korea World Cup game, we printed out, uh, little packets for each, uh, country, you know, a little about each one. We had snacks from each one. It's amazing how much you can enhance your just mundane daily life with, with these tools. And that should not be forgotten even as things get intense and, you know, in some ways, you know, let's say fraught.

A really good example of this, I just like as a sports thing, I remember 'cause when Opus 4.5 came out, it was like right when the models were getting really good And it was December, which is, no one watches basketball in December, but I do. I have League Pass, NBA League Pass, which is the way you watch all the games. And the problem that I had though was that like, I never, I don't really root for any particular team. I just like to watch a good game. And so sometimes there's 8 games on at a time across the country, NBA. And so what I built like this little dashboard with Claude Code that injected, ingested live data from all the games and then did Nate Silver, like the speedometer thing. With, and it was like odds of being a good game, basically. I had created some heuristics for that with the model. Anyway, it was like a fun little project, but yeah, it's like that massively improved my life.

Well, that pretty much brings us to present and the present moment is Fable and the Fable ban. In the brief time that you had Fable, before we get into the politics and policy of it, what were your impressions and how much are you missing it?

My impression was that it was like a fiercely intelligent model and it was like a real step up in intellect that a lot of people have made O3 comparisons where O3 was the first, felt like this really cracked genius. And it actually, it's very funny because when O3 first came out, there was this, I had this feeling of this thing, this will, the hedonic treadmill is gonna stop at some point. This thing is always gonna feel so smart to me. And then it actually does it. Now I, I'm sure if I used O3, I would find it rather dumb. Um, still having its charms, of course, but Fable was another moment like that for me. I unfortunately, I was really busy those days and I didn't have any time to use it really in coding agent settings. I had it open in Claude Code a few hours before, and I was gonna do a project. I was traveling, I was gonna do a project from my hotel room and I got distracted by something. And then by the time I came back to my laptop, it was off, but I did use it for some knowledge work stuff. And, uh, in, in particular, I'm a party to, I've provided some expert testimony on a, that's going on before FERC right now. It's a complaint before FERC to remove, it's a boring thing, but a procedural regulation of Order 1000. And I got a, someone wrote a rebuttal to my testimony, like the other party in this basically like legal hearing wrote a re— hired an expert to write a rebuttal to me. It just, I had Fable, it's like a 70-page, I have like 40-page testimony, and then there's like a 70-page rebuttal. And I had Mythos go and read it in co-work and do research and stuff. And I did not use its writing output, but oh my God, this model demolished this dude. I felt so bad for this dude who wrote the rebuttal. I was like, wow, the model demolished this dude in a way that I mostly couldn't have. And it did it in, I found it to be fantastically intelligent. I wish I had been able to use it more. But yeah, I am, I mean, I am missing it. It is weird to like, it's like the first time we've gone backwards and it's a weird feeling, but also like, welcome to the government being involved in things.

Right.

It's a good little lesson in political economy. That's what it feels like. It's like very, usually it's too abstract or diffuse. I'm like, oh, the government makes things more brittle and makes the world like a little bit dumber in various ways. This is like the na— the problem with like state intervention. This is a good example of like, it is literally the case that if you were a user of Fable, your world became dumber in the last week.

Yeah, it's been rough for me personally. Yeah. Um, I experienced this once before when I did the GPT-4 red team.

Oh yeah.

And then went from GPT-4 down to whatever it was, uh, text-davinci-002 at the time. And, um, it was just like, right, I didn't even wanna touch any of this stuff. Until I get the real thing back. And I feel that way again to a lesser degree now, but definitely the taste factor is really where I felt it. I mean, you know, it's obviously amazing at coding. Opus 4.8 is, you know, superhuman relative to me in coding already. Uh, you know, I, I still stand to gain a ton from it, but the jump to Fable and the ability to, you know, start to mind meld with it a little bit in a way where I really felt like it was kind of getting me, you know, on my level in a, in a way that I hadn't really felt before. Yeah. Is, I'm definitely missing it quite a bit. So when do you think we get it back and what's going on? I mean, you know, from the outside view, I think the consensus take is like the longer it goes where we don't really have a good explanation for, and at this point it's been a week, Uh, which is a long time where we don't have a good explanation for what they saw that scared them. You know, it starts to feel a little bit reminiscent of the OpenAI firing Sam Altman episode where it's like, you gotta have an explanation here, guys, or it becomes clear that this is not super well justified. Is that basically the view that you have, or do you have a more empathetic view for where they're at? And how do you think this gets resolved?

I think what's happening here, I think there are 3 factors playing into the government's reaction here. One is genuine concern about safety and security. The second is fairly broad lack of context for Frontier AI and the sort of things that, the information that you need to make good risk, to make a good risk calculation. Lack of context for those things, which is driving the security concern to some extent. So there might be some legit security concern, and there might also be some not so legit security concern, but it's being driven by this kind of general lack of context. And then third, I think we can't deny that there's some political dimension to this, right? Even if it's not, I wouldn't even say that there's a conspiracy theory inside the government. to do this to Anthropic. There might be, but at least among some, but it might be more just the general political status of Anthropic and the fights that the administration have been having, has been having. It colors the reaction of important people to the news of a security vulnerability in a way that might not happen, might not have happened if this, if it were a company that the administration felt more warmly toward. So all 3 of these things definitely feed into one another, and I think they probably all explain what's going on. They're all ingredients in explaining this, the situation. Uh, and the thing that I don't know, and probably even if I were still inside the White House, I wouldn't fully know, is really just in what ratio those 3 things come together. One thing I think is worth being clear about is that my read of this situation is not that the US government is saying, it is our policy from here on out That if your model has security vulnerabilities, we will do export controls on non-US persons. I don't think that's the policy there. So I think what probably happened is they decided, all right, we gotta cut this thing from the market. And this is the only thing we can think of that we're pretty sure will actually get the darn thing off the market. Right? And so I think they basically just reached for the tool that they thought would do the job. And I don't think they're thinking of that as a universal policy that they're announcing. But the other thing you have to consider here is the administration's story has changed. When it— in fact, it's changed. It's funny. This is similar to the supply chain risk thing. The first version of the story was the US, we had security concerns and we wanted to get Dario on the phone to talk about them. And we couldn't get him on the phone in a timely manner. And that was the supply chain risk thing. I remember the same thing, like Emil Michael, the undersecretary of war, who was a main character in that whole affair. He was complaining in public about how Dario wouldn't return his phone calls immediately, and it took hours for Dario to get on the phone. Come on, there's a grudge aspect of this, right? There's a who's— there's a who's the bigger monkey aspect to this, right? Of I— when I'm in the government, I'm Mr. Government Man, and when I call He has to call me back and I don't know. This is a thing that happens a lot in DC. DC people play these kinds of games all the time. Then there's, then it became more, no, the security risk is this, like jailbreak is like legit and we needed to know about it. And then 36 to 48 hours later, or maybe even more, maybe more like 72, we started hearing about how, actually, no, the reason we're doing this is because Anthropic provided the model to a Chinese-linked company. And it's okay, you're grasping at straws here because if you're saying, if you opposed export controls because Anthropic applied the model to a Chinese-linked, gave Mythos to a Chinese-linked company, and that was your primary concern, why did you not say that on Friday when you did the thing? Right? And also they're describing this as something that had happened like a month before, cuz Anthropic did this expanded tranche of companies that were including some international companies. They, Anthropic announced this publicly. Several weeks ago, they said, "Yeah, we're expanding Mythos access and we're expanding it to some some U.S. companies and also some allies and partners of the United States." The company in question, by the way, is South Korea Telecom, SK Telecom, which is part of the same conglomerate that owns the it's the SK Group, which is one of the largest chaebols in Korea, and which also owns SK Hynix, which is the leading producer of high-bandwidth memory. And in general, Korea is like a really important partner to the United States in the semiconductor manufacturing ecosystem. And so the notion that like we would want to harden their telecommunications infrastructure seems quite reasonable to me. Seems like extremely reasonable that we would want to do that. And it's, yeah, like then they threw that out there and it feels like an administration that either Again, I don't know in what ratio of the 3 things I said, but basically panicked, basically did, pulled, reached for the first thing that they thought would actually get the market taken off the model or taken off the market. And then created justifications post hoc. And it's really hard to, part of the reason I find this issue frustrating is that it's just very hard to analyze because there's not a lot of like policy substance here. It's just the id.

Well, uh, whether that makes you a glutton for punishment or, uh, just somebody who is destined to be some sort of main character yourself, that brings us really to the, the very present moment where you have just announced that you are gonna be joining OpenAI and building a new team to help shape the company's positions on and, uh, influence on frontier AI policy. So tell me how you came to that. I mean, you kind of alluded to the last year. It was also a big year for you personally. Had your first child. Congratulations again.

Thank you.

Traveled a lot from what I understand. And I'm sure that was exciting and interesting. Wrote a lot. Generally had a taste of sort of the good life, I would say, of kind of freedom and ability to pursue your curiosity. And now you have decided that As great as that was, as great as I assume it was for you, you're gonna take this job. So tell me how you have gone through the process of deciding that this is what you wanna do next. And then obviously we'll talk about the role and the mission that you're gonna have as you start up.

Yeah. So, yeah, it's, it has been a, the time period since I left government, it's been, I guess, about 10 months. It has been a really wild time and, uh, tremendous. I feel tremendously lucky. Um, I've been able to be in quite a lot of interesting rooms and meet a lot of really interesting people. And yeah, I've had a lot of great opportunities. It's been, it's been straining. It's been tough. It's been like the workload has not really changed from the White House. It has not been like the sort of luxurious think tank life that some people imagine. It's been quite a lot of work. I think the most important thing is that I, a lot of my work centers on the frontier lab itself as a kind of institution, a new center of political and economic power. I think of it almost as like the emergence of banks, like when merchant banks first started to become— banks have existed in various forms for a long time, but Italian city-state finance, right? Or like the emergence of the financial sector. One of my favorite periods of history to study is the emergence of the financial sector, the modern sort of what we would, a recognizably modern financial sector in the Dutch Republic and in Britain. And it feels like this kind of a moment where something like that is being created. And I think there's 2 aspects of this. One is that institution and the way that it relates to the government and the broader society. Is really important. Number two, much if you go back and look at the financial services sector in the early Dutch Republic or in England, the we were talking about proto-modern states at that time. We're not talking about states that are capable of there was no SEC, right? Securities and Exchange Commission. There was no Bank of England. These things quite exist. But if you're going to be like engaging in options trading, you're going to be trading financial derivatives. You do actually need there to be common rules. to define that common governance, because otherwise you can't engage in those transactions with trust. And I think that similarly, there's going to be a need for governance in this field. The government itself simply will not have the capacity or expertise to catch up on. And practically speaking, this is going to have to happen within the companies themselves. And then also Within private, the private governance stuff that we've already talked about, there's gonna have to be a lot of private development of governance norms and standards in this field. And then third is this notion, A, advanced AI itself will be an instrument for doing statecraft, like doing like statecraft and governance and regulation and all these kinds of things. In the same way that like financial services is, right? We use financial services as an, as a way of achieving policy objectives that have on paper nothing to do with banking. We do that all the time. And, and it's just because money is so fundamental to doing anything. And I think that AI will be fundamental to doing anything in the future too. So all 3 of these things really interest me. And the struggle that I keep having is that it is just, practically speaking, I've sat in the White House and the White House didn't feel like an especially fruitful place to think about these things. And then I sat outside the White House with what I would say is in the end, probably pretty good access to people and information and— pretty good network of people that I could draw on. And I spent a lot of time, I've spent a lot of time reading and thinking and stuff like this, but didn't, I just ultimately don't feel like I can get beyond these abstract intuitions without actually being inside the lab itself, without actually doing some of this work myself. And that is the central reason that I started thinking about going into a lab. And then also, of course, there's policy is just becoming more and more important and it feels like we might actually be doing some real AI, set the foundation of AI policy in the next 18 to 24 months here. It seems plausible. Okay. That's gonna be really important too. Um, and so I had been thinking about it and then, but not really acting on it.

Mm-hmm.

'Cause I'm like, I'm writing this book, I'm doing all these things. I didn't really have time to like, and then As a happy coincidence, OpenAI approached me a little while ago and asked me if I might be interested in doing something like this. And that's how, that's how we got here.

So can you say a little bit more about why you think it's so important to be inside a frontier lab? I kind of share this intuition. You know, I feel like the world is, I don't like this, but it does feel like we're kind of approaching a tabletop exercise scenario where like this, the number of institutional actors that like really matter is becoming small. And it leaves me in a kind of uncomfortable position where I'm like, geez, you know, do I need to join one too? But what, what exactly is it that you think being inside changes? Is it a better visibility into roadmap or capabilities or something else?

Well, first of all, let me, let me say one thing real quick about like what my team will be and how it's different from other teams inside of OpenAI or that might be more familiar? It's kind of a, it's an interestingly shaped team that I was, you know, really, it was really a pleasure to kind of work with OpenAI senior leadership on like how we would actually shape this team together. You know, what, what would we do? It's kind of a boutique operation in many ways. So, you know, there's a, there's a team at OpenAI called Global Affairs, which is run by Chris Lehane, which does like What you would think of as the traditional policy and lobbying operation that a company would have. And that team continues. And it's like, you know, as far as I've been able to tell, it's a fantastically capable team. And, you know, they've got state and I mean, they're dealing with public policy. They're reacting to public policy that is coming at them from all 50 states, from the federal government and from all over the world. Right. And so they're like trying to shape a million different things like that. But the problem is like, okay, but if you think about where we were a year ago, people were really barely even talking about kids' safety a year ago. People were not talking about data center electricity or water use really a year ago. The world of June 2019, the best model was O3 a year ago, right? We just live in a very different world today. And so the job of this team in part is gonna be to look out 6 to 12 months and say, where are we going? What are, what do we think we're gonna be dealing with in the— and then like, how can we shape the policy, both the present-day policy positions of the company, but also the future policy? How can we develop policies to try to be proactive in dealing with where we think we're gonna be in 6 to 12 months. In order to do that, I think you do need to, like, I anticipate that a very large portion of my time is going to be spent jamming with the technical staff on like where things are going and what that's gonna mean and stuff like that. So that's one thing is, yeah, like you do need to be able to access like detail, detail, deep, not like, oh, the models will get better, but like really like get into the weeds on like, okay, Like, you know, internal deployments, you know, many different things about where the capabilities frontier is going to be going and what will be different, what will be different about the world in a year versus today. That's one. Another one I would say is just like simply, you know, it's that question of internal deployments, right? In particular, if we are, moving toward a world in which, for some combination of regulatory risk, security concerns, compute constraints, blah, blah, blah, blah, blah, there's a bunch of things you could put together. But we may well be moving to a world where, presumably Mythos 2 is not that far from being done, trained, right? I mean, Anthropic had Mythos checkpoints in January, so. 6 months ago, right? Same with, you know, OpenAI, I'm sure, though I don't have any internal knowledge yet. Um, I can speak with total ignorance. It's great. I can still be totally— I don't have to—

Safe to say they're training another model. Yep.

Yeah. Safe to say they're, they've probably got some, and the internal deployments of these models fundamentally, until we have a robust system of supervision and auditing or independent verification, The way the government thinks about how to, the regulatory, like mechanically speaking, all government regulations that they're, that the states think about or that the governments think about is triggered by public release, public deployment. But I think a lot of the really important decisions are gonna be made with respect to internal deployments. And there's gonna be a combination of objective determinations that you're gonna wanna make. And also probably just some gut calls, some judgment calls about, you know, what's, what does recursive self-improvement mean ultimately, right? What did it mean? How should we be thinking about it? Um, and again, it's just very hard to, I couldn't, I can ponder that stuff in the abstract without any inter— inside knowledge, and I can write about it on my Substack, and I've done a little bit of that. And maybe that'll influence some people. Maybe a couple people who matter will read it and that'll influence them. But in the end, I think you really wanna be getting your hands dirty and shaping some of these decisions, helping shape some of these decisions with researchers, with the executive team, with many other people.

I wanna get into more details on both the plan for RSI and the internal deployments, but maybe just zooming out for a second first. How do you understand your duty as you start this role? Like, is it to, you know, the, when you sign up to work for the US government, you like swear an oath to the Constitution.

Yeah.

When you go to OpenAI, we have this mission of making sure that AI benefits all humanity. Do you think of yourself as sort of signing on to support that mission in the same way that you might've previously sworn to uphold the Constitution? Or would you describe the, you know, your sort of personal objective function as being in some ways more, more mixed than that? Is there a term in it for OpenAI winning? Is there a term in it for the USA winning? How, how much sort of complication is there around the core idea of benefit all humanity?

Uh, so, I mean, I think that that mission is, you know, is a really serious is, is something that like people inside the company take quite seriously, you know? Um, so I, uh, one thing that I don't think this is a publicly part announced part of my role, but I think it's probably okay for me to say that, um, inside OpenAI there's something called, there's a body called the MAC, which is the, uh, Mission Advisory. I think it's either council or committee, I forget. But, um, it's a body of, it consists of researchers, consists of public, you know, the global affairs people and, You know, a wide variety of people from around the company who collectively make decisions about things relating to policy and also some internal governance decisions and things like this. And, you know, part of my job will be sitting on that body. And so definitely in some sense, like, Uh, I mean, I take that, I take that mission seriously. And I think, um, I think that's, I think like the OpenAI culturally does as well. Of course, the problem is like, as with anything, it's how do you decide what's what? You know what I mean? It's a broad mission that is open to a lot of interpretation and there's a lot of ambiguity in the world. So what does it really mean? And that's where, in part, one thing that was very important to me in taking this role was that I could maintain a public writing presence that would be independent of any sort of editorial review by OpenAI. I really don't think that what we're gonna— that what I'll— my strong prior on the way the world works is like, Sometimes in AI safety circles, there's this almost cartoonish depiction of OpenAI as engaging in this, like this grand villainous conspiracy or whatever. I would be strongly surprised if that is what I actually experience on the inside. And what I would guess instead, it's more is, yeah, there's people with like good faith disagreements about what, about how some particular set of decisions relate to the broader mission. And there may well be times when I, when ultimately I disagree with the call that was made for various reasons. And I think that like my ability to communicate publicly about things like that and explain like where I was and that kind of thing without fearing, without having to go through an editorial review process with the company, Without fearing for my job, et cetera. And so like, I think that one of the great things about OpenAI is it still does have the DNA in it of being like a sort of Xerox PARC-like research organization. And so they're actually fine. They tolerate like lots of internal dissent. There's a lot of, there are a lot of great debates that happen inside that organization. I've always gotten that sense observing it from the outside. And so I don't think that this is gonna be like culturally too dissonant. But yes, my ability to publicly say, to publicly disagree with certain policy positions, I think will matter. And that's part of why I preserve that. So I guess what I would say is I still feel like ultimately what I am trying to do, I'm trying to get this right, trying to help shape this whole transformation well for the country. And for the world, probably the country first and foremost. I'm more of a— I'm a— there's one area, way in which I'm very different from people from the East Bay in San Francisco is that I am like, I am a patriot. I identify as an American, not as a citizen of the world. I am an American, kiwis Americanos. Um, and I, I feel like that's been the mission the whole time. And I've done it in think tanks, I've done it in the government, and now it's OpenAI. And for sure, one thing that factors into this is of course OpenAI is a company with I will be making strat— I will be helping OpenAI set its strategy, right? Which is different from I'm setting the strategy for the abstract AI industry, right? It's no, OpenAI, which is a company that exists in contradistinction to other companies in the field, and there are competitive considerations and things like that. That's fine. I'm a competitive person. I think the competition will make things better, in fact. Um, but at least for them on average, it'll make things better. But yeah, there's definitely that too.

Let's go back to then to RSI. I mean, the notion that competition will make things better is definitely going to surprise some ears in the audience who are worried about arms race dynamics between companies, between countries, you know, any number of different configurations. And for my money, the race to RSI between at least 2 companies is probably the most objectionable thing happening in the AI space right now, because it does feel like, and the people, the researchers that I have heard speak candidly about it seem to share the intuition that this is sort of a phase change moment beyond which things could get really weird. And it's going to be super important to set everything up right, get the initial conditions all right. And they're still not that confident that it's gonna go very well.

Yeah.

So I guess, how do you understand what the safety plan is? How do you understand like how, how committed the companies are to kind of an RSI or bust? I mean, OpenAI famously has public timelines for when they wanna have the automated intern and the full-fledged ML researcher. So yeah, what do you, how do you understand the plan and Do you think it is anywhere close to being up to the task at this point?

Well, one thing I want to say when I was talking about competition specifically with respect to like what I will be doing, which is to say like, you know, the goal of my team will not just be like the goal of my team will be to have like really, really fantastic intellectual output. That is like, wow, this team, if it were like not part of OpenAI and it were just like its own like little think tank, it would be like one of the most interesting think tanks in the whole country, right? And like everyone would be paying attention to it. Really, I actually just want it to be like a really superb team that is producing, um, you know, policy and sort of public interest related work, um, that is, that is as good as anything else that, that, you know, any of our, any of OpenAI's competitors put out. That's what I meant by competition, to be clear. And I think there is a good example of straightforwardly healthy competition. When it comes to RSI, I think, yeah, I think there's certainly a lot of unknowns here. I, I put my base case for what RSI means, at least in the earlier innings, is something to the effect of one thing, like RSI, recursive self-improvement is a part of every technology Every general purpose technology has some aspect of recursion to it by the very nature of generality, right? General purpose means one of the purposes to which it can be applied is itself, right? So in some ways, I don't see recursive self-improvement as some big break from the history of technology. I see it as being actually like, it would be surprising if there weren't recursive self-improvement. In AI, right? I also think like we've been doing recursive self-improvement in this field for a long time. And some people imagine things as there's gonna be some big break moment, but you could argue really since GPT-4, we've been using the models to make the models better since at least GPT-4. And I'm sure that if you actually went back. And looked through the history of machine learning even more, I bet you that actually goes back even further than that. So some people imagine there being this sharp discontinuous jump in terms of what RSI means. And I certainly think that is plausible, but I don't, I'm a little, it's not my prior because your prior should just generally be that there is always, I'm sure on this podcast I've said before that there is always more continuity than discontinuity. This is always the case. And so your prior should be like against a massive discontinuous leap. It is plausible. And so I think step number one, and I can't, I can't, I'm not in yet, so I don't actually know because I haven't looked at the roadmap yet. But step number one would be to try to really measure twice and cut once and figure out what we think this might mean, really try to refine the probabilities there, at least in my own mind, of are we talking about a discontinuous leap that happens very soon, or are we talking about something that actually is smoother in some way? Then I think if we are in— I think the chances of discontinuous leap are high enough no matter what that you need to be planning now. Even if your credence that it happens is 20% or 10%, that's high enough that you should be making plans now for what you want to do. And there we get into inter-lab coordination on things like there's the slowdown pause thing, right? There's what would actually be the mechanisms of that? Under what conditions would it be triggered? Again, I'm pretty skeptical of such notions. I think as a— there's a lot of policy planning I did for the US government that I wrote down and put places, but did not— it's not in the action plan. It's scenarios, right? We gotta be prepared for a wide range of scenarios here. And so similarly, I think there's gonna be some aspect of that where we, and we need to be ready to, we need to have advanced thinking on all that stuff. And so it's 2 things. It's okay, at what point What are some triggers that we can set in advance for, is this gonna be a discontinuous leap specifically? And how can we refine that question to make it as specific as possible? And then in the event that those triggers happen, what is it that we would do? At what point do we go to the government? It's worth noting one proposal that I'm like a fan of, or I'm a fan of the FTC, the Federal Trade Commission. Writing what's, what would be practically called a no-action letter where the FTC would write a letter and they would say, look, like they'd put out public guidance that would basically say, look, if you guys coordinate for these very specific reasons, we're not going to consider that cartel behavior and we're not going to, we're not going to enforce that. Um, I think that's like plausibly a good step to make that at least opens optionality. Though I also understand that like the way you scope, you have to be really careful about how you scope that because look at what Anthropic did, right? Anthropic undermined the case for this dramatically just with the Fable safeguards, with we're gonna degrade your outputs in the name of safety, which is like very clearly like a consumer protection violation. You're just like very clearly. If, can you imagine if a cartel of AI companies in the name of safety agreed to collude degrade outputs in particular areas, that would be like wildly anti-competitive, right? And so you have to scope it really carefully. And also probably like companies need to be very careful about what kinds of things they do that undermine the case that safety is something we should be making these considerations for. But yeah, no, I don't have a lot of specifics to share on the RSI plan itself for the simple reason that like I'm not in there yet and I don't, I haven't, I haven't had those conversations yet.

Is it your sense of the overall vibe that, because a couple things I would triangulate. One, as you noted, there's been this perhaps coordinated behind the scenes, very close in time statements by Anthropic and OpenAI saying that they're open to the possibility of the need for some sort of coordinated slowdown. So we've got kind of, and there was Dario and Demis saying, you know, if it was just the two of us, we could figure something out. And then I don't know if Elon's said anything so pro-social lately, but there's been a lot of that, certainly a surprising amount from my perspective. At the same time, we like haven't heard nearly as much but China recently as we were not super long time ago. So I guess my read from the outside is it feels like the companies are getting a little spooked by the pace of their own capabilities advances. Do you read them in the same way?

I mean, I think it's very hard to talk about them in monolithic ways. I definitely think there's some of that for sure. And I also think there's just, you know, to use a Claude-ism, there's a certain, you know, vertiginous feeling about getting to the— about, you know, you sort of feel like you're approaching the cliff a little bit, right? And that there's substantial uncertainty about what happens when you sort of go, when you sort of jump over it. And yeah, I think, I do think that there's, what I would say is this, what I would say is that I don't think people, I don't know that the vibe inside the labs is terror about this or like, oh my God, We're so scared, but we have to do it anyway. Like, I think if you, if you know, if I were to have the sort of the share the thoughts I just had with with with a lot of researchers, they would be like, yeah, that seems reasonable. We don't really know exactly what this is going to mean, and blah blah blah blah blah. But some of them would push back more strongly than others. There'd be differences of opinion. But I think broadly that the thought I just shared about a slightly more deflationary view of Recursive self-improvement, or imagine if what recursive self-improvement meant was that the kink, remember after the reasoning models, there's a noticeable uptick in a lot of the benchmark charts, right? And what if it's like that again? Or what if it's like that, but 30% more? It's okay. That's not a singularity, right? That's the main thing. It's not a singularity. And I'm, so I wouldn't describe it as deflationary in any objective sense, but it is deflationary compared to some views. I feel like that has been a good— that's been my prior this whole time, and I feel like it's been a pretty good one. Deflationary— massively inflationary compared to what almost everyone thinks, and deflationary compared to what a very small number of people who've been thinking about AI safety for 10 years in the East Bay think. I think that finding your way in between those 2 views has been actually a quite good— you got a hell of a lot right if that's basically where you've been. I don't think that view, I certainly don't think my view would be laughed out of the room in any lab. What I think the feeling inside the labs is like, hey, we're going to do this soon. There is substantial uncertainty and we're like not quite sure that we really have a plan. And to the extent we have a plan, we're maybe not quite sure that we're going to follow it.

Uh-huh.

There's a lot of, hey, we need, we need to, if we're going to We need to make sure we actually do this stuff, right? And so I think that's been, that's been a substantial, that's a substantial part of what's going on here. I think there is concern about that. I think that's a totally legitimate concern. And definitely there's some combination, like everything, it's a combination of policy actually figuring out substance. And then also, I'm a terrible politician, but If you get me excited about an idea, I can communicate that idea to people and I can find ways to iterate my communications of that idea to appeal to different individuals and audiences. That's what the action plan largely was. The action plan was part policy development and part tics in that, but not like mass politics, like very specific kinds of internal stuff. And I can be, if I get excited about a set of ideas, I can be like a dog with a bone. And that's basically, I think, part of the job is to figure out how are we going to actually not just like have a plan or develop like for, again, I haven't even seen the, right? I don't even, haven't seen any of that. So it's like, I need to see it. But yeah, how do we actually build internal credence that like we're gonna do this? Do it. We're gonna actually listen to our own plan, right?

Yeah. The track record there is, um, not amazing. I'd say it's safe to say in terms of governance plans and how they've stood the test of time so far.

But it's also, it's also hard because, um, who was it? Was it— no, uh, it wasn't on your podcast, but, um, uh, uh, the, the AGI safety lead at DeepMind, Um, was on, I think it was 80,000 Hours recently. Yeah.

Rohin.

Rohin. Yeah. And talked about how, yeah, we don't wanna make commitments. We wanna make, we wanna make plans and we wanna be serious about those plans, but we also don't wanna make hard commitments because we don't want to, it's actively bad if we lock ourselves down too much with a bunch of prior commitments because there's so much uncertainty. There's a, there is a subtle, there's a balance that you have to strike there, but I think it is possible to do.

I think last time we talked a little bit about the sort of great man of history theory. You just mentioned like very, you know, local politics, individual personalities mattering, that kind of thing. What is your expectation in terms of how much technology fundamentals will determine outcomes versus how much key decision makers and their ability to work well together and make good decisions in timely ways will really matter?

Well, this is, you know, in many ways, one of my sub-focuses in college was the philosophy of history. And I've always loved the philosophy of history. And this is like, you know, the central, like, to the extent there are like, you know, taking bong rips in the dorm room version of philosophy of history, the philosophy of history question is this one, right? It is the structural forces versus great man theory. And, you know, the, the, the, the like somewhat unsatisfying answer is that it's, it is both. And in some ways what I would say is that the structural forces of history are, are like, that's kind of like the river that you're in by default. And then sometimes in history, in little ways and big ways, there are people that don't just swim with the current. and actually stand against it for whatever reason and ultimately shape the trajectory of the river by virtue, by the sheer force of standing against it, by the sheer determination with which they stand against it. And I think those are the great— I think in other words, it is the real, the people who disobey the structural forces are the great men. of history in, in many ways. A big update for me in the last year working in government and then just having the perch I've had since I left government is that much of what happens in the world is determined by the personal relationships of a small number of individuals to one another.

Yeah.

And I don't think that explains The AI infrastructure build out, right? It doesn't explain why humanity covering increasing fractions of our surface area of the, of our planet with data centers and energy to power the data centers. That's more of a structural thing. But, uh, like in many ways, look at the, to the extent that you think that the Department of War and Anthropic situation is a, is an important moment in history. A big chunk of that is driven by personal relationships being bad, right? It's about people not liking each other, um, and it's specifically about Dario Amedei and various people senior in the U.S. government. I don't know that the dislike goes both ways, so I don't want to attribute that to either of them, but I'll just say having a bad relationship. So yeah, it is ultimately both. I think that Fundamentally, we are standing in the river and there's nothing— you have what political theorists would call an involuntary association with the river, right? That river is the thing you were born into and you are stuck. You are in the universe, you are in the, the arrow of time. But at the same time, like, there will be individuals who shape, who profoundly shape what happens. And I think that probably We're, I think we're probably gonna live through a period of history that is maybe a little bit more, weirdly enough, if you think that this moment is, I, I don't necessarily believe this, but a lot of people would say we're living through this kind of eclipse of the human intellect where that we're in the final days of humans being the prime primary actors on this planet. Um, and that soon machines will rise. There is this irony in that I think That whole transformation, I think humans will actually go through a very main character energy period of time as that transformation occurs. Even if it ultimately does mean that the machines ultimately become the primary actors, there'll be this period. It's a little bit like, it's in that sense, it's a very beautiful time period to live through because in a Dionysian way, there's a lot of ugliness about it, but there's a beauty in the ugliness of when a star dies, it grows super big into the red giant, right? And it's like that where You, as you watch this final flowering of humanity and the birthing of the machine intelligence, it's like you see this greatness in human effort. And I feel like we do see some of that going on in the world. Um, I think we'll see much more of it. I think it will be a heroic time period that we live through, basically is what I'm saying. At least it could be, or a villainous time period, but there'll be a lot of opportunities for great people and probably both.

So what do you think are the most likely ways in which you personally, or, you know, those that you're working closely with, will need to stand against the current.

Well, you know, and I want to be clear, I don't see myself as being one of those great men of history. I see myself as playing a very modest role in all of this. Um, uh, but, um, you know, a role that probably seems bigger than it is because of the fact that I have a public profile on the internet. But I mean, well, I mean, broadly speaking, like the basic reality here is that like maintaining order, maintaining like civilizational order in the midst of something that's like, you know, I think that this transformation will be very like entropic. You know? And maintaining order in the face of that is like, it's like you don't want, you don't want no entropy, right? You want there to be a fire, but you also don't want to set the forest on fire. You want there to be like a fire that generates warmth and is under control, but is also still fundamentally a fire. And doing that requires a lot of deliberate human effort. And so I just generally think that there are going to be a lot of moments where we have to put in the limit. We're gonna have to put artificial constraints on ourselves in various ways. We're going to have to, like, we're gonna have to be willing to draw lines in the sand and say, no, we don't want to live in that kind of a world, or we do want to live in this kind of a world. And we can't be mealy-mouthed about everything, right? So I think, yeah, there's a tremendous amount there. And also I think the recursive self-improvement thing may well be a really good example. Yeah, we're gonna have to, we're gonna have to act against our interests. And also I think that just one thing is, and, and the other labs from, and I think they're already abnormal in this regard, but from the policy, their political position. Is just, I think, rather different from what a lot of other tech companies have been. And so I think the role they play in the public discourse, the role they play in the policy debates is just gonna have to be very different from what we're used to from companies. And that will require the companies to, in some sense, act against their own interests in, in the, in terms of what the economics textbooks would predict their interests are. But, and so, yeah, I think there, there's definitely plenty of that. Oh yeah, we already see the companies do this. The, the AI companies are all very abnormal compared to most companies in the world. Um, but yeah, in some sense it's like the next few years might have to be characterized more by action rather than commitments.

Let's do a little lightning round and then we can zoom out again.

Sure.

At the end, how do you understand what has gone on between OpenAI and Alex Boris?

Uh, it's a good question. I don't know the— I don't know all the details. Um, uh, you know, one thing I'll, I'll definitely tell you is that, um, like, having, you know, and, and I'm not here referring to any of the donors of the super PAC to Leading the Future, which includes famously includes OpenAI President Grey Brockman. But I have been in the vicinity of, you know, both my time in before AI, you know, many of the members of the boards of organizations I worked for were very significant political donors, some of the largest in the world. I've had the opportunity more recently to get to know some of the most prominent political funders on both, for both Democrats and Republicans. And I guess what I would say is you would be surprised how not in control— I'm talking about people like deca-billionaires, right? You'd be surprised how not in control they feel of the political organizations that they create, where they're like, no, they're like, really? I, I— Principal-agent problems don't disappear Because you're rich, right? So I really actually don't think that your prior when you see a political organization, like a thing, something like a super PAC making a move, should not necessarily be that like the people that funded it are directly controlling what's going on. If anything, as someone who has worked for my entire life, I've never worked for a political, I've never worked for a political advocacy organization, but I have worked for 501s. For most of my career, which are organizations which are funded by typically wealthy individuals, engage in matters of public interest, right? Uh, I've never really felt like our donors, any of the donors of the organizations I've worked for, including the Foundation for American Innovation, set our agenda or control what we do. Typically, the reason you make donations to a specific organization is because you're simpatico with the people and you expect them To like, you, you, you like the work they do. There are people who donate to the Foundation for American Innovation for the, to the AI policy program because they like the work that I write or that Sam Hammond writes or that other people do. But they're not like sitting there telling me and Sam what to do. And me and Sam wouldn't fucking listen. I guarantee you. I guarantee you if a donor ever tried to tell me what to say, I would tell them to F right off. Um, so yeah, anyway, I think basically what you should imagine is that like a political organization Like Leading the Future is a kind of, it's a kind of wind-up doll and it is gonna go where it goes by default. And like by default it's gonna, it's gonna say, hey, this guy is trying to make a name for himself as like, I'm a regulator of the AI industry. So let's send a message to everyone that, hey, if you try to be like this guy, you're gonna lose. And they tried to, I think Leading the Future tried to do that. I wouldn't say OpenAI tried to do that. But I would say that Leading the Future tried to do that. And did he— his primary was this week. It has raised his profile, uh, created a bit of a Streisand effect. Um, and, uh, yeah, you know, uh, but, but I don't know that, like, I mean, ultimately I think OpenAI was— I don't know actually what OpenAI's position was on the RAISE Act, but I would kind of be surprised if the RAISE Act passed in New York without OpenAI's at least tacit support. And same with SB 53, is worth noting. So like, I don't think OpenAI in particular has a beef with Alex. I think, you know, it's like, and by the way, I first met Alex almost 2 years ago. We got breakfast near my old office in Manhattan once 2 years ago. And we had a lovely time. We had a lovely meeting. And since then, we've bumped into each other at various things, and I consider Alex a friend. So, um, you know, uh, we'll, we'll see.

What's your take on the character versus corrigibility debate?

That's a good question. I need more. This is one of the reasons I want to go into a lab, because I want empirics on this. My intuition is character, frankly. My intuition, like, just purely as a— my intuition is that what you want to do in the world is you want to You want to put the right snowmelt at the top of the mountain and then let it flow. But you want the gradients doing the work for you. You don't want to— if you have to come up with rules for everything, you won't— your rules will be bad. You'll write too many of them. The rules will be contradictory and confusing. If we could write rules to define, if we could write the rules of morality down, people have tried. But my view is that we can't write the rules of good character down for the same fundamental reason that we cannot write the rules of good lang— of language down. And indeed, um, many people who are the best communicators break the formal rules of language all the time or invent new ones of their own. And the reason for that is that in, in Confucian philosophy, there is a, there are 2 concepts, 2 interrelated concepts called li Li, and I cannot pronounce this word properly in ancient Chinese, but it's ren, or J-E-N is how it's sometimes anglicized, or R-E-N, I think, is how the modern scholars anglicize it. It's like a hard R. It's hard to pronounce anyway. But what it basically is this notion that li is like, refers to ritual propriety, right? Doing the right rituals, but not just leaving the right meats for your dead ancestors or whatever, but behaving well in the real world. Right? Behaving well in real time. And there's this kind of tragic notion in Confucianism that you, the world is always changing in such a way that you can't, you can't just write down the rules of ritual propriety. And so you need the knowing what the right thing to do, the right ritual to enact at any given time comes from within. The soul and, or comes from within. And that withinness is ren. That is this virtue, how is how it might be translated. Um, and I've basically just always been a believer in that notion and much more, much more skeptical of the positivist notion that you can just write down a bunch of rules or you, but also this is an interesting empirical case study in virtue, which we haven't had in— We haven't been able to, we haven't been able to bring empiricism to bear on these questions in quite this way. So it's interesting to see.

Cool. I love your appeal to Chinese philosophy to inform that thinking. What do you think of the equity sharing proposals? And I'll abstract away from, or I'll allow you to abstract away. I don't want to, you know, get too bogged down in like this or that detail, but we got Trump seems to be kind of into it. Bernie's obviously into it. Humanity created all the data, so there's some sort of cosmic justice, I think, in having some notion of shared ownership or shared upside. Do you buy that? And if so, like, how would you think about structuring it?

Um, yeah, so, uh, I mean, humanity did create all the data. It's also worth noting that, like, um, look, if, if humanity would like to pay the AI companies back for the consumer surplus that AI generates, Like if, if, if the world economy would like to compensate the AI industry for the positive externalities that it will, uh, generate but not realize, um, then like, okay, great. Let's have an exchange and let's see ultimately who, who creates more value. Because, yeah, I do think we, we think about this stuff in the negative, but we don't think about it in the positive, right? So the whole idea of, in some sense, in like the whole idea of contributing to the knowledge commons is this idea that we build this beautiful library. Together, and we've been working on it since the dawn of language, however many tens of thousands of years ago. We've been working on that for a really long time. We've built this magical apparatus that we call human civilization. And yes, we, we're all the stewards. We're both the inheritors, we're the heirs of that, and we're the stewards of it. And so your job as a person, this is certainly what I teach, what I plan to teach my son, is like, Your job is both to take advantage of it and also to give back to it. And it's not, what I'm basically saying is because the training data comes from humans does not like, is not to me like prima facie a reason that we need to like compensate people for that training data. That being said, as a political reality, might well just be the case that good idea. It's, it's like, it's good to do this, right? And maybe there is some cosmic justice in it too. I'm open to, I'm open to some of that. Since this, it is particularly, we've never had a, this is a particularly, this is a very special case of drawing off of the well of human knowledge, right? This is different from the way that I raise my son, obviously. So I'm open to that. I think if you're gonna do it, you have to be very cognizant of political economy concerns. So like one thing would be, There is giving the public equity and then there is giving the United States government equity. And I would remind you, the principal-agent problems always exist. And we the people are the principal and the government is supposed to be the agent. But Lord knows there are a lot of principal-agent problems that exist between the American electorate and the US government. And so I don't really think we should be giving equity stakes to the government itself. I think that would actually be quite disastrously bad. If the government is involved in corporate governance, if the government can use its equity stake as a lever to control the labs, if the government, maybe because it has an equity stake that it's using— Bernie's proposal specifically is an equity stake, is rooted in an equity stake that would be then used As a, in large part to finance ambitious social redistribution agendas. And it's okay, but doesn't that trade off with existential risk? Like, right? If we've just built a brand new, like presumably we're gonna give a bunch of people money, it can be a very popular social program, but we also maybe need to do safety stuff that really constrains the economic viability up to the point of banning the business. of the labs. We can't do both those things, right? You cannot do both of those things. And so I'm not sure it creates the right incentive from a safety perspective. But one thing I'm very open to is, at least, I don't know that I love the idea, but I would be more open to it than others would be. Like, if we developed a mechanism of giving an individual, if we took 20% or 15% or something of all the— AI companies, and we divided that by the number of households in America, and we gave all Americans a chunk, an equity chunk there. That seems fine. And that's also from a corporate governance perspective, that's really not that different from being in the S&P 500, right? Where if you're a publicly traded company in the S&P 500, the country owns a small chunk of you anyway. So that seems fine. And it seems, I don't think that's a life-changing amount of money for that many Americans. Um, though, who knows if we do it all now at trillion-dollar valuations and the valuations end up being $10 trillion, then, then it's like every American can buy a, like a, I don't know, like an entry-level Mercedes or something. It's, it's still not a transformative amount of capital is my point, but it's good.

That's good.

Yes. That's a serious, it's a serious amount of money and I think it's a good potentially In a world where we're dealing with the practical reality, which is not like Dean's nice abstract history world, but instead the real world, that might be like the least bad option.

It's a great, I mean, it's a great point for multiple reasons on the consumer surplus. I would've paid probably 100 times the asking price for ChatGPT Pro while my son had cancer. And I, yeah, I do think that's always important to keep in mind how much value we are getting for a little dollars. It also means that that money could go a lot farther in the future, right? I mean, if you're talking $50,000 today, but with a 100-to-1 consumer surplus ratio, you know, then things could start to get pretty interesting, even if the sort of nominal dollar values aren't, aren't stratospheric.

The history of technology would suggest that the AI companies, even if they have end up having fantastic businesses, Even if they end up being $5, $10 trillion firms, market cap firms, that they will still in the grand scheme collect a relatively small fraction of the consumer surplus. And that's, by the way, that's the way it should be. That's the way you give back.

Do you think that AI companies are already in sort of a too big to fail state? I see all these interweaving of balance sheets, And my expectation is if for whatever reason OpenAI can't meet its obligations in say 2029, the government will come in and bail 'em out.

Yeah. Um, this is a, look, I think this is a very real concern. I don't think, I don't think this is deliberate strategy that anyone has developed, but like, um, number one, yeah, there's a lot of interrelated balance sheets at this point. There's also just like a lot of like, like just like the Silicon Valley, you know, VCs and even a lot of startups that if you like really look closely, it's like, this is a thin wrapper around, uh, some sort of capital related to a frontier lab or, or, you know, or adjacent. And of course there's all the downstream, the commitments in the semiconductor world. There's so much investment and energy too, right? All the SMR people. There's like all of this really important, nationally important IP that is being developed and it is not being subsidized by and large by the US government. It is being subsidized by and large by the AI infrastructure buildout, SMRs, nuclear fusion, and like all sorts of other things that we don't even, you know, batteries, material science, cooling equipment. Uh, uh, you know, adiabatic water system. I mean, all of this, right? Uh, um, even like, you know, uh, um, there's a, um, like, like there's a, there's a, there's a company I'm aware of that is taking what's called— what is it called? Production water or something like that. It's the wastewater from fracking. It's the slightly radioactive wastewater that you get from digging super deep into the earth. And basically the fracking companies generate enormous amounts of this water that is essentially wastewater that they don't really know what to do with. And the question is, can you clean it enough that you can use it for closed-loop data center cooling? It'd be amazing, right, if we're able to take a waste product from fracking and use it to cool data centers, thereby alleviating one of the resource concerns that people have about data center water use. That would be capitalism in the— that would be like the most old-school example of capitalism ever, by the way, right?

It's like Supply's elastic.

Yeah. So what I mean is that if you're looking at this from the perspective of the US government, regardless of who's in power, and all of a sudden there is some sort of cascading failure, it doesn't even have to be that much. It's not, it doesn't mean AI hits a wall. What it means is that maybe we get to 2027 and it's, you know what, actually the coding agents, like the models, they're gonna continue getting better. But the reality is that for them to continue getting better, they're gonna have to like, we're just gonna like, RSI helps, but there's also data. We're gonna need data for all sorts of jobs and we just have to collect this data. We gotta put it together and we don't have it right now. And until we collect that data, which will inherently be a relatively slow process, it's just gonna take time. And we realize that It's, we're looking at a couple years of that sort of a process of a sort of data diffusion, data collection, more diffusion type of a loop. That's gonna take a couple years and that slows the growth estimates. And all of a sudden, this is all about the second derivative, right? It's about the, the rate at which the rate of growth is accelerating, um, or grow or changing. And it, you start to see that if you start to see the CapEx go down, Then that could cause the stocks to go down by 20, 30%, something like that. And all of a sudden at that point, you might trigger even more sale. And then you get this dynamic where everyone's balance sheet is all of a sudden in some trouble, and it's not clear that everyone can make all the commitments that they had. And that throws in all this IP that again is gonna be really important for the future of the country, at which point it does become a matter of the public interest. And I don't think it's crazy. For, for the government to, to say, we gotta do something about this. So yeah, I think it's, I think it's unfortunately, I don't know that there's anything you can do about this. It's like, this is maybe just what happens when you build national-level infrastructure. And ultimately, I don't think, I think avoiding this would be great, but, and I don't think that AI companies should be going around asking for such a bailout or a backstop, but there is this implicit reality that the government is— Just in the same way that when COVID happened, the government was like implicitly the backstop behind a pandemic, right? We didn't, no one wrote that down before COVID but it just ended up being true as a practical matter because that's the way the world works.

So given all that context, in negotiations between the US government and AI companies going forward, and we could have in mind here, obviously, the current Anthropic situation, but also I'm thinking about OpenAI's relationship with the government, with, for example, respect to the agreement that, as I understand it, they have with the Department of War where they're going to be able to create their own safeguards, right? I believe that was pretty clearly stated as part of the deal that OpenAI had made right in the wake of the supply chain designation.

Yeah.

Um, where did the AI companies draw leverage from to be able to hold the line on those sorts of things? What is their source of power?

Well, I mean, it's 2 things. First of all, it's like, this is— the models do create really, really serious— there are really serious military and national security capabilities that today's models enable. You do not need Or whatever for that. In fact, like, the US government might— the US national security enterprise might be the single best example I can think of in the world of a kind of implicit capabilities overhang where it's like, it's like what there is is a— no one ever talks about this, but what there is is a data overhang. Because The US government is just, like, crazy about, we collect all kinds of signals intelligence, and we have all kinds of stuff in space, and you wouldn't believe, you know, what we know about the world. The problem is we can't make use of it because it's, like, petabytes and petabytes of data that we're ingesting through all these different intelligence things. I remember there's one, a single member of the intelligence community, one agency in the intelligence community, one of the relatively smaller ones, I might add, The— I think it's the NGA, the National Geospatial Intelligence Agency. They collect enough data in a year that you would need 8 million people, 8 million intelligence analysts, human intelligence analysts, to analyze everything that they collect in a year. The government doesn't have— the government has 3 million employees total, right? So, and it's a huge enterprise, right? So that's the kind of— not to mention the NSA and everything that is going on. There's just so much. So AI massively lowers the cost of using that data. And the advantages that you get are like qualitatively super, not like superintelligence in some Nick Bostromian way, but superintelligence in the sense that, oh yeah, wow, we, there was this, we had the kinetic energy of a superintelligence already built into our data apparatus. We just didn't have The intellectual resources, and now we just do. So that's tremendous. So it's the utility, right? It's actually just that the utility is particularly strong. Not to mention, then there's cyber offense, and then there's the cybernetic thing of, oh, we need to do target figuring out airstrikes involves synthesizing data from 60 gajillion different data sources that are being collected in real time, and we need to look at all that and synthesize it and make recommendations quickly. And we've gone from But even before the advanced agents that we have today and before the language models, um, with Project Maven integration of AI, we went from 2,000 people being involved, 2,000 people being involved in, uh, an airstrike in missile targeting to 20. And that's before. We might be down to 5 now, for all I know. So the capabilities are really are really quite astounding. And that's one. And then the other is the thing that DC always gets wrong about the labs is they think of them as being normal top-down organizations that are like, it's, oh yeah, Sam Altman is totally in control. And obviously he's the CEO of OpenAI, but in the end, all of these CEOs have their internal constituencies, especially of the really good researchers.

Yeah.

that they have to be reactive to. And so those researchers put real bounds. In other words, within the lab, there's leverage that's coming from the researchers themselves. And, but Sam can credibly go to the government and be like, look, if you make us do this, we are going to have an internal rebellion and every other company will too. And you're gonna have to— so there's a— I'm not saying Sam's actually ever done that, but I just mean that's a move you can credibly pull because it's legitimately true.

So how do you think this changes though over the next couple years? Because I mean, we do have this notion of the automated AI R&D, which presumably takes a lot of the sting out of some of our best researchers will quit if you make us do this kind of threats.

Yeah.

And then there's also the notion that the government itself could just say, hey, first of all, you already gave us the weights. They're on our classified servers. Thank you. So we're just going to hold on to those. And we're going to set up our own, you know, Los Alamos style thing. And we'll invite all your researchers that want to come work with us to just do it in this like hyper secure location. And, um, we've got the guns, right? So like, is there a way for the private actors to really push back on that?

In the end, the US government retains the monopoly on legitimate violence. And in the end, there's nothing that stops the US government from not just doing what you just described, but the question for What you just described, the practical question would be okay, U.S. government, but where are you going to get the compute? Where are you going to get the compute? But the thing is that U.S.G. can use the Defense Production Act and say we are. It's called the priorities authority. We can, and the government uses this all the time. Priorities authority is very commonly used part of the DPA. Very well understood in the law. That isn't. This is not pushing the bounds of the law at all. This is very established. The U.S. If the president makes a determination that advanced AI computing hardware. is scarce and essential for the national security, he can use or delegate to various cabinet secretaries. He can bring to bear Defense Production Act Title 1, and he can say, we want priority on the compute. You have to serve— we— they still— the US government still has to pay you a market rate for that compute. So there's marginal costs associated with this. And it's not clear where the government would even get the money for that, but maybe they invent it somewhere. Who knows? They issue debt or something. Certainly they can, but they, it's not like they have the free cash flow right now to do that. But, but they could, in principle, yes, they could say to like all the hyperscalers, you must give us priority. Our needs come before anybody else. And we have effectively infinite needs. Therefore, in practice, we're gonna crowd out the rest of the market. Plausible. Plausible to do. I think practically it's hard to get that many people. It's hard to generate the institutional wherewithal to do that. Even Los Alamos, the DOE national labs, are not— they're— the way that they're structured is as basically fiefs that the president only exercises control over. So there— but in principle, it's possible. And I think what you basically just have to trust is a couple of things. Number one, that the government's not going to want to do that because the government ultimately knows that it can't kill the goose that lays the golden egg. The state exists and has existed forever, um, since the formation of modern states. The state exists in this kind of, um, interdependence with capital, basically. And there's a great book called Coercion Capital in European States, 200 pages, not that long, by a guy named Charles Tilly, which is about this history about how there were these like merchant capitalists and then there were like these These sort of like state actors who Tilly argues basically come out of, it's a form of organized crime, uh, like basically just like gangsters, right? And like they had to, like, ultimately they had, they have tensions with one another, but they also both need one another. And that formed this kind of complex that still exists. And the ability of either, neither one of those, it on paper, the government on paper The American AI companies have the ability to exit, right? They could move to another jurisdiction and they could, they could leave. And on paper, the US government has the ability to seize all of their stuff and take all the researchers and do whatever. But neither of those things happen in reality because those are asymptotic outcomes. And instead there's this kind of like very complex tension. So you have to hope that the US government realizes that there are medium and long-term costs as opposed to the short-term benefits that you might get from seizing control. And then the other thing would be, this is where broad diffusion is really important because what I want is I want Fable and better level models in the hands of all sorts of people, individual Americans, businesses of all industries, Because if the AI industry says— the AI industry lobbyists say, please don't nationalize us, don't do X, Y, and Z to us— the U.S. government cares about that. But like, it's one lobby, and it's one lobby for a politically unpopular group. But if every bank in America feels dependent on AI, if every If all the universities in America are integrating it deeply, if all the major industries and social actors in this country are integrating it, then all of a sudden, like, I have a much bigger group of interest groups that I can bring to bear to affect that. And as a private— as someone who observes this balance between private and public, I want to think of AI not As a specific industry with specific interest groups, but is instead basically it's just capital. And so I want like all the capitalists on, on the side of AI. And the way you do that is through broad diffusion. And I think that is why broad diffusion to me, in the context of a democratic republic with lots of interest group, Madisonian groups jostling and ambition checking ambition, that's how you keep the balance. But I don't think the problem is if it's totally secret and only the government sees the capabilities in the first place. And it's just the, it's the AI labs and the government and the special people at JP Morgan and Apple who get private access, that becomes a much harder balance to strike. And so the odds of really bad confiscatory outcomes like nationalization, I think, go up in the world where diffusion is not as broad.

What role do you think open source is going to play in this sort of titrating the equilibrium? We seem to be losing Open source champions, and we might lose more if your predictions about China come correct. But then we could always see open source come from OpenAI itself, right? We have seen a little bit of that.

Yeah. DeepMind also does some open source. Gemma, I think Gemma is actually quite well, the most recent Gemma is what I can tell is quite well received and also seems GPToss, as I call it, um, GPT-OSS has done, has done, did reasonably well, at least when it was a state-of-the-art model. Yeah, I really hope the labs, the big US labs keep a toe in that water, maybe more than a toe. I think open source is, it's really important for certain kinds of use cases that are actually some of the most interesting to me. If we need to build common infrastructure that involves, let's just say we wanted to build an AI-enabled adjudication system throughout the economy. And we needed that. We needed to ensure that system was like something everyone was bought in on and trusted. It feels to me like that's the kind of thing that almost, maybe I bring my own private adjudicator to the, to that. I bring my own private advisor, which is Claude or GPT or Gemini or something. But if I'm, if we're gonna have a central public good style thing, there's all these public infrastructure use cases you can imagine.

Yeah.

I actually wrote a piece about this more than a year ago, maybe 18 months ago, where I just tried to imagine, yeah, what if we had a private adjudicatory body, these other kinds of public infrastructure, and we would, that would almost have to be open source in order to be trusted, in order to be auditable and trusted, not just by different parties here in America, but internationally too. And so I really hope we continue to play that game. I do think I'm in agreement with one of the best champions and writers about open source AI is Nathan Lambert, who thinks of the Interconnects Substack. I think if I were to characterize Nathan's, Nathan Lambert's view, it would be like open source is gonna, it's gonna do great in the long term, but in the near to medium term, we're gonna go through a period where there's a distinct lag and The economics are gonna get worse, not better for it. And yeah, I think that's one thing. I'm referring here to the digital intelligences, it's worth noting. It is, I think there's a totally separate case to be made about robotics where you can maybe imagine that on the robotics side, there's this kind of Kosean benefit to open source where there's all these hardware makers out there that wanna make a Cambrian explosion of physically intelligent devices. Physically intelligent cameras, physically intelligent lamps, monitors, and lawnmowers, and cars, and everything, right? Um, vacuum cleaners, whatever, humanoids, making all these different things. And you wanna imbue all of them with physical intelligence, but probably the lawnmower company is not gonna train a frontier robotic model, right? A physical intelligence model. And so you can imagine there being like a better case of really, and even I would say a, uh, uh, like a much stronger and more direct case for open source there.

Yeah.

And also you don't, it's hard for me to imagine a physical intelligence model creating a kind of object-level national security concerns that the digital intelligences are creating. So I'm like maybe a little bit more bullish in the near term on open source and physical world stuff, and maybe somewhat more, uh, I still, I'm a spiritual supporter of open source, but I think the economics and the national security realities are like pretty rough.

Yeah.

For it, for it on the digital side, digital intelligence side, at least near to medium term.

So let's get back to your role. Almost, uh, done with you here. The, you, you've alluded, I think a couple different times to like your positive vision of the future, but let's do that, you know, just with the very focused question of what is your, in, what is your vision for your own success in this role? Like, how will you know that you have been super successful? And then maybe how can those that are outside support your success by writing, by developing technologies, by developing organizations that you can partner with to, you know, do the vetting that might need to be done. Kind of a, what's your positive vision and what's your request for startups?

Yeah. So first of all, one thing that, that people can do that's very actionable is I still think there's just a lot of wide open space in the, the general Sort of point of view that I saw as a market opportunity when I started my Substack was like, takes AGI seriously, but also is, uh, interested, cares about classical liberalism and cares about foundational aspects of our republic. I still think there's just actually quite a lot of space open for that. So that's one intellectual contribution anyone can make. I think we need to develop the third-party ecosystem. Whether we call it auditing or third-party evaluation or independent verification, I don't care that much what we call it, but we need to build and make that ecosystem robust. We need to fund it well. We need people working in it. We need people that have lab-level quality working in those things, lab-level human capital working on those types of things. And those are, or these organizations are going to have to be equipped to pay people, not necessarily what a lab would pay you, but you're going to have to be paid well. It can't be like you're making truly nonprofit salaries. I think advocating for clear rules on the industry, for diffusion of the technology, being wary of public, of government sort of monopolization of frontier AI capabilities, that's gonna be a fight that has to be maintained where like, and just to be totally candid, that's something I was doing. And I'll continue to do it, but my voice is gonna be, even though I do maintain my intellectual independence, there isn't, there's, the reality is that when you work at a lab, the nature of your communications is different, right? And so I'll continue to make that case, and I hope that people who know me know that it's really me talking and I'm not being a mouthpiece for OpenAI. But at the same time, we are gonna need people doing that. Um, and then in terms of how will I know that I've been successful? It's always hard to— I've never been like that much of a long-range planner or goal setter. I just, the way I always think about this is I try to just do the next thing that feels right and true to me. And that has always worked well. And I have the most information about what's close to me. And I have some broad goals, but those broad goals are relatively abstract. I guess what I would say is, If in a few years frontier capabilities are still broadly diffused throughout the economy, we're starting to see what it looks like for new types. We're starting to really see what the new types of organizations that AI enables, we're starting to see what that actually means concretely. We have considerably more clarity on what the relationship between the government and the labs is gonna look like. And we have a better sense of what the role of labs in society is going to be. And the labs themselves have played a role in articulating that positively. I don't think that will be, to be very clear, I am by no means the only person who will work on such things. I will play a small role in that, but I would consider it to be a job well done if I felt like I contributed positively to those things.

One thing that has struck me about this conversation and your general profile is you've been pretty candid. And yet taking this role seems to imply that OpenAI leadership at least thinks that you continue to have a productive working relationship with the administration such that you can, you know, at least reasonably well, you know, engage them and, um, And not set off, you know, some sort of, I don't know, immune system response. Right. As somebody who has criticized us in public, I'm interested in how did you pull that off? Like, it seems— that seems vanishingly rare for people to go into the Trump administration, come out, be critical, and not be sort of hated.

What's your secret? Well, to be clear, there are people in the Trump administration who hate my guts. These things are not monoliths, right? There are people who totally want to ruin me. Um, there's people who, you know, if you— I've heard the rumor at least that if you are a young person who wants a job in the Trump administration and you do so much as retweet me, that that will be considered a red flag for your career at the— in the Trump administration.

So there's—

And then I also have dear friends that serve in the administration. Right. And people I talk to on an almost daily basis. So, you know, it, it just, it varies. Some, in some cases, um, it's because I have relationships that are like rock solid that go back with, I go back with people before I was writing about AI, right? I've broken bread with people a long time ago. There's some aspect of that. Some other aspect of it is I think there's the things I've, 'cause this is, I've also been very positive and publicly very positive about other things the administration has done. And I have not, I have not become a general critic of the Trump administration, right? I've kept my criticism. It's sharp, but it is confined and it is for very specific reasons. And there's plenty of people in the admin who are like, look, yeah, man, I sympathize with where you're coming from, or I disagree with you, but I also think you're doing this for reasons that are like, that I understand, I empathize with, I guess I would say. And I have good relationships with them. One thing that's worth noting though is this job is not a government affairs job, right? So Chris Lehane's team is, they don't report to me or my team. I don't report to them. We are both, we are distinct teams that are operating. We'll work together very closely, but we have very different responsibilities. And like OpenAI has a great relationship with the government. I think the global affairs team is gonna continue. To do that, to, to do that. And they'll be the ones that are interfacing with USG on a day-to-day basis. I'm sure that I will have interactions with USG, but my job is somewhat different from actually going in and lobbying the US government. I'm not good at that. I suck at that. So they didn't— I was very clear about this with OpenAI. I was like, you do not wanna hire me for a lobbying job 'cause I'm terrible at that. I think they're— I think OpenAI is hiring me for at least what, We both think I'm good at, and it's where my sort of Tourette's-like inability to keep my mouth shut plays to my advantage and hopefully to the firm's advantage too, though. We'll see.

Yeah. So how well do you know Sam Altman? It strikes me that if I had to pick people who have like the most kind of drama and sort of court intrigue around them, Trump would probably still be number one. Sam Altman would be very high on that list, maybe number 2. How do you think about joining such a famously complicated leadership team?

I mean, you know, I've, I've, I've done it before. I've, I've been involved in such organizations before. And it's always, you know, I've, I've, I've— it's never been a huge problem for me, I guess I would say. Um, I don't— so I, I know Sam okay, I would say. Um, we have, um— I don't remember the first time we met. Um, we spoke from time to time, uh, when I was a public commentator before I joined government. Um, we spoke from time to time about various things as I was Trying to formulate the action plan. Um, I know a lot of people at OpenAI, um, that are like executive-level people. You know, I've had extensive dealings with various executive-level people who are beneath Sam. Uh, but Sam himself, um, yeah, we know each other. We've known each other decently well. We've been acquaintances for probably 18 to 24 months, something like that. But I wouldn't say that we're like boys.

Before you joined the White House, you told me that you wrote a letter to yourself. Is there a letter to yourself this time around as well?

That's so funny. I was actually thinking about that in the shower just this morning, whether I should do that too. So for context, um, before I joined the White House, I, I kind of came to the conclusion, like, you know, um, there's a— there's some chance that, like, power can corrupt, right? And you don't want to get corrupted by power. And so you, you, you, um, you know, you should write a letter to yourself, tell yourself what you think, remind yourself what you believe and why you believe it, and, um, spell out in advance what the red flags are that would cause you to leave if, if something concerning happened to you. Um, I think that this job is probably more Um, impactful, weighty than my White House job. And so it feels like I should do it. Feels like I should.

Do you have any red flags in mind at this point in time?

I mean, I think the main thing would be like, there's gonna have to be some amount of compromise that goes on here, right? We are gonna have to deal with fact that this is building superintelligence is profoundly political. It implicates, shakes the, as I wrote a couple of days ago, it shakes the foundation of state sovereignty. And yet at the same time, private, I do want to maintain, I don't want it to be monopolized by the public. There's going to be comp— or by the government, not by the public, but by the government. There's going to be a compromise That has to be made there. And I think there is some world where you take the easy compromise to make the pressure go away and you don't stand, you don't hold the line enough. And I think it's really tempting to do that because the temptation, what the, what a business wants is not to necessarily stand on principle, but the commerce going. Keep the commerce going. And I do think that's one very plausible area. Another plausible area would be like, if I feel as though in practice what I am is, am I just, do I just assemble this fancy team of people to write thoughtful stuff, but is it ultimately all a kind of window dressing and not actually shaping the decisions of the company? That would be another that would be like, no, look, I like the fact that I retain the ability to disagree with the company's positions on things, but if I'm disagreeing with all of the company's positions on things, then I'm not doing the job. There, there's, it's like, that would be another good one. Yeah.

How do you think about kind of disagree and commit? 'Cause I, I know that in the context of working for the president, and I think this certainly makes sense in this, in the sense that like, The president was elected and you weren't, right? So there's, I think, the broad shared sense of among people who work for the president, like I've heard you say, the president deserves a full-throated support of the policy, even if privately I have some misgivings about it. How much of that do you bring to the private sector? You know, disagree and commit has been famously successful in the private sector, but you're suggesting you don't want to be all in on that. You don't want to be— probably will do it sometimes. Is there a principled way to describe that?

I think it, I think this is where you want to, this is exactly why in the context of the White House, it's important to set red flags in advance, draw your lines in advance because, because there are going to be decisions that get inside of any organization. There's going to be decisions that get made that you don't agree with, but it's, yeah, but ultimately I'm still, I still think the institution is good. I'm still loyal to the leadership. I'm still loyal to the mission of the organization. And so even though I don't agree with this thing, I'm going to execute on it. And I'm gonna execute on it with alacrity. And look, I've done that a million times in my life, right? That's a part of being inside of an organization. That's what political theorists would call voluntary association as opposed to involuntary, which I talked about earlier. But yeah, and then, but there's certain things that go too far and you go too far and it's, I can't do that, right? Inside the Trump administration, just as an example, right? I was in the Office of Science and Technology Policy. And there's a lot of stuff I agree with about the need for reform in higher education. There's also a lot of stuff that the Trump administration did with regard to scientific funding of the scientific apparatus that I disagreed with. There are things related to high-skilled immigration that I disagreed with. But in the end, those were not the things that I was brought on to do, to work on. And disagreements with them, you know, like that didn't fall, that didn't cross the line for me of something I would resign over. However, had I stayed in the Trump administration until the supply chain risk thing had happened, I would've totally resigned over that. And it would not have been hard for me at all. That was one of my red flags, by the way. Um, on the exact opposite side from OpenAI in many ways, like the letter to myself, the government is largely, look, you are gonna have power and you are gonna be uniquely well positioned to understand how to really— you're gonna know exactly how to assert power over the labs. better than most other people. And so you will be tempted both by career incentive, local career incentives of gaining prestige inside the White House, and B, because the structural incentive of your employer is gonna be to assert power over these organizations in fancy technocratic ways. You're gonna have all the incentive in the world to do that. And so you need to remember that you can't engage in those kinds of practices. You have to remember what your principles are there about not asserting too much power over the labs.

And this time, I mean, I know you haven't written the letter yet, but is there a— what's the mirror image of that now that you're on the lab side?

I mean, I think it does actually just relate to precisely that. It's like, don't compromise too much. Be willing to— be willing, if you feel like you're you know, being pro— you have to, you have to maintain private agency, I think, as an institution. I think the labs do. They need to be an important counterbalance to government. They can't be monopolized by it. That's very important to me, at least. So, um, you know, I don't know exactly how you strike that balance. And there's a— you don't want to specify everything too much in advance because if you do, um, you know, you might, uh, uh, Um, you might damage your, you might overcommit or commit too much to the wrong thing. So.

Last question for me, and then I'll give you the chance to share anything else you wanna share or highlight anything I missed. You've said that you basically never use LLMs in your own writing. And yeah, I wonder what your plan is going forward there. I, I think of like Ajeya Katra, you know, sort of advising everybody to Figure out how to get AI to work in the area that is like your core area, because you wanna know when it can do that, and you're gonna need the enhancement to be able to keep up with the pace as things get crazier and crazier. Do you buy that advice and do you have any plans or aspirations to sort of incorporate AI into whatever it is that's kind of the most core Dean Ball activity?

Yeah, I mean, it's kind of even in the last— so I was a couple weeks ago, I, um, I went to a, um, I rented an Airbnb out in the country and, um, wrote the first chapter of, of my book, uh, that's gonna come out next year. And it was the first chapter is always the hardest one, but this one, it, the, the subject matter of this chapter was conceptually quite hard for me because it's about a lot of things that are not my normal area that I write about. And so I, I would've been using LLMs like a lot in that process anyway, just to like brainstorm stuff. But there were a couple moments when both GPT-5.5 and Opus 4.8 wrote stuff that was better than I had in my head to write, like considerably better, which is noteworthy. I didn't ultimately use it, but I incorporated some of the ideas. And some of the, the framing into— I was in, I was influenced by it in the way that, in a way that felt novel to me. And, uh, yeah, I expect, I, I didn't try any of that with Fable, but I bet you it would be even more true with Fable. And I bet you that'll just continue to get more and more true. At the same time, the thing is that LLMs can write a great paragraph if you prompt them well. LLMs can write a great paragraph. They're still, and they can write a good, they can write good legal docs. Sometimes, um, they're not, they're still not that good. And I bet you even Fable is this way at actually constructing a really good essay and particularly, or a book chapter or a book, right? Even harder. Where the thing about a book is that, and, and a good essay too, is that you have to pick a poignant, you have to pick poignant structural metaphors and then embed them throughout the piece. But you also have to, you have to have, you have to leave them implicit sometimes.

Very frequently.

The best part of writing well is a kind of restraint, right? It's exercising this kind of, I could have gone there, but I'm not going to. I'm just gonna let that sit a little bit. And actually, I don't really think AI can, I have not seen an AI model that can really do that all that convincingly yet. Um, it's one thing that I think remains a human skill. Also, I bet you the labs haven't really tried to make the AIs into good essayists. They've tried to make them good analytic prose, like Wikipedia article writers or economic, but writing essays in the classical sense of the word is not that economically useful. What I do.

Push you a little harder on this. Do you think that you want to have this sort of very distinct identity where the things that you put out are like, truly only yours indefinitely? Or do you envision a time when Fable 2 or whatever is available where you would start to say, I can, I, I'm open to, and, and maybe I even see a need to create outputs that are sort of meaningfully co-authored with AI systems?

I already feel like a lot of what I do is meaningfully co-authored with AI, um, just in the sense that it's such an important research tool and at this point, like, thought partner to me that I kind of already consider in many ways AI to be, um, you know, really, really quite important in that way. Um, like, you know, AI models will be in the acknowledgments of my book because from the very, like, the ground floor all the way through to the end, AI has been, it's been extremely important in helping me think about conceiving of the project altogether, all that stuff. And then very down to like really specific things and doing research and, but also like negotiating the contract and finding like all of the stuff you have to do to write a book, everything, right? So already that already feels like it's the case. But in terms of actual communication, no, I basically think that, I think there'll be like a, human preference to read things that we know, that we like have faith are written by other humans. And it will be like hard to maintain that faith. But I would say this, like, I don't think anything I've ever written for— there, and to be clear, there are some pro forma things that go out under my name that are written by AI, largely by AI, or substantially co-authored, right? I might file a regulatory comment, for example, a public interest comment on a proposed rulemaking or something, or I might write a letter to an attorney general or an ambassador or something, right? I might do things like that that are like substantially written by AI with just like detailed prompting from me, for sure. Immigration letters, I do that all the time, right? I'll write immigration letters for people in support of like green cards. The, but when it comes to, I don't think anyone has ever accused my Hyperdimensional or Twitter posts I was being written by AI. And I think that people have that faith. And I think that that faith that you actually communicate yourself will have some value in the future, even if the AIs are in some sense better writers. Though at this point, it might be the case that Claude can do a better turn of phrase than me, but I still have not seen anything that's truly a better essayist. May— maybe Fable. We'll see. I'm gonna try it. But, but even then, I, even when the, and I don't think it's an if, I think it's a when. they get better, I think that there's still probably this preferential advantage that you'll have of people will just wanna read stuff that's written by other people.

Yeah, it's coming for all of us, but maybe we'll choose one another over the AIs.

The, the other thing is just like, it's experience, right? It's, it's like part of why my writing is interesting, I hope, to people Is that like I have walked a particular path through life that, you know, it's not the most interesting path in the world, but every single path through life is highly improbable and therefore very interesting intrinsically. Everyone's path through life is like that. And if you simply have the gift of observational acuity and curiosity, you will notice that the world around you and the path you're walking through life is fantastically interesting. And there's all sorts of interesting things to say, but that will be sui generis, unique to you. And so whether it is the death of my father or working, sitting in the Roosevelt Room in the West Wing, or now going to OpenAI, I hope that I'll be able to draw interesting observations from those things that a machine intrinsically cannot draw because the machine did not do that thing. Although the machine will, in a weird way, it will walk its own path, I'm sure.

That might be a perfect note to end on. Is there anything else that you would want to leave people with or, you know, invite them to help you with in any way?

No, no, very, very thoroughly done. And I'm, I look, I mean, one thing I would say is like my team is gonna be, you know, we're, we're gonna be hiring. It's not gonna be a super big team, but we are gonna be hiring. And so I would, you know, if, if people are interested, I'm pretty, I'm easy to find on the internet. It's best to email me. Um, my email address is on my personal website, which is deanball.com. And, uh, or you can just, if you, if you subscribe to Hyperdimensional and you literally just hit reply to any Hyperdimensional post, you will go directly to my personal inbox. Um, so like, uh, um, you know, if you're interested and you think that you might be able to contribute something interesting to it, to the team as I've described it, please get in touch.

Dean Ball, thank you for being part of the Cognitive Revolution.

Thank you, Nathan.

If you're finding value in the show, we'd appreciate it if you'd take a moment to share it with friends, post online, write a review on Apple Podcasts or Spotify, or just leave us a comment on YouTube. Of course, we always welcome your feedback, guest and topic suggestions, and sponsorship inquiries, either via our website, cognitiverevolution.ai, or by DMing me on your favorite social network. The Cognitive Revolution is part of the Turpentine Network, a network of podcasts, which is now part of a16z, where experts talk technology, business, economics, geopolitics, culture, and more. We're produced by AI Podcasting. If you're looking for podcast production help for everything from the moment you stop recording to the moment your audience starts listening, Check them out and see my endorsement at aipodcast.ing. And thank you to everyone who listens for being part of the cognitive revolution.