AI 2025 → 2026 Live Show | Part 2

Bye. Next up, we have, uh, Congressman— no, New York State Assemblymember Alex Boris.

Future—

You're getting just a little ahead of yourself. Yeah, with Congressman there.

Future, future congressman, hopefully. And he is a computer engineer by training. He was at Palantir for, uh, 4 years. I am definitely glad that we have engineers now entering public service, uh, which hopefully will at least make our leadership cohort younger and more tech savvy. He is also the author of the RAISE Act, which is now— I think the president is trying to preempt with his executive order. So there's lots to talk about. Alex, great to have you on the show.

So glad to be here.

Cool. So welcome. First question, you sponsored the RAISE Act. We did a whole Cognitive Revolution episode about it some months ago. It has. At the time, it honestly didn't seem like it was going to be a super controversial lightning rod sort of thing. You know, it seemed like it was less that than SB 1047 was. That has not necessarily held over the intervening months.

It's true. Where are we?

I know there's been sort of, you know, let's say, edit to the bill from the governor. Now we have an EO that sort of targets bills like this. And then we also have super PACs that are coming online to target people like you for bringing these bills forward in the first place. But just for starters, like, where are we in the, in the lifecycle of the RAISE Act?

You've hit all of the major ingredients. So the main aspect is that we're in active negotiations with the governor right now. New York has a fairly unique process called chapter amendments, where after the legislature passes a bill, the governor can negotiate with the sponsors of the bill for changes that she would like to see. And If we reach an agreement, she signs the original bill but attaches a memo that says, I'm signing pursuant to this agreement that we've reached. And then we, the sponsors, will introduce all of the amendments early in the next session. About a third of the bills that get signed in New York— it varies year to year, but roughly a third of the bills end up going through this chapter amendment process. We always knew raise would be a part of that. And so we're actively in that process now. The governor Or I should say the bill was delivered to the governor last week, and the Constitution gives her 10 days excluding Sundays to sign. So that means she has until Saturday to sign it or veto it. Although the message that we're getting is they're trying to shut everything down by Friday night so people can actually have some holidays. So in the next 48 hours or so, we should know if we have agreement on a revised version of the RAISE Act or if unfortunately we don't.

And what are the kind of key outstanding points of negotiation right now?

There's a lot. I don't want to get too into sort of conversations that are happening because you want people to have the freedom to express, hey, what if we tried this? What if we tried that? So I have to stay somewhat above the specific details. But there's been a lot of reporting this week that there's a lot of pressure to move to something that more matches SB 53. Versus what was initially in RAISE. You know, curiously, there hasn't been a push to match the European Code of Practice. So it seems like it's less about maybe standardizing everything that has to be done versus weakening it. But the point that I keep making is that there's no company that would be subject to— if we were to pass the exact same bill as SB 53, there's no company that would be subject to it in New York that isn't already subject to it in California. So it's not like passing a data privacy bill where each state that passes it, you give new rights to the citizens of your state. If you just reaffirm the exact same bill, you haven't changed anything. It's like being the 39th state to ratify an amendment to the Constitution, right? 38 is enough. It's in the Constitution. The 39th that does it doesn't make practical changes. And so we're having real conversations about, you know, SB 53 was always meant to be the floor. Are there steps we can take that advance AI safety that we can all agree on? Or are we, are we destined, unfortunately, to have to try again next year?

Or both?

Let me take a step back and like, what do you think are the dangers that are trying to be addressed by the AI safety envisioned in the RAISE Act?

There's a number of them. And to be clear, this is not— there's a lot that we need to address with AI. Whether that's its effect on kids or the environmental conversation, or its effect on democracy or the workforce, right? Discrimination—all of these things. The Raise Act in particular is focused on really extreme risks, on the type of catastrophic risks that would signify something going really, really wrong. It's a set of risks that should be addressed at the federal level, but frankly have not been. And in fact, it seems like most of the push is the other way about stopping the federal government from taking action. And certainly. there's a push to stop the states from taking any action. But because the federal government hasn't, states have been stepping up. And there's 8 states that had some version of a frontier model safety bill this year, and only 2 managed to pass it. And California had a very, very strong version that then SB 53 became that compromise at the end of their session that is focused mostly on transparency. There aren't really safety standards in the bill, but there are transparency standards.

And that's very important.

And then the RAISE Act, which passed in New York, which put real safety standards on what could be developed and requiring the labs to have a safety plan that they make public and actually stick to, to disclose critical safety incidents, things that show a massive increase in risk of something bad happening. And if a model fails their own tests, that they shouldn't release the model. And that's designed to stop the situation that we had with the tobacco companies, where they were the first to know that cigarettes caused cancer but continued to deny it publicly and release their products. We're saying in, in many cases, we recognise the labs are the experts in what's happening. They are at the frontier, and if they discover an extreme risk, they have a duty to act upon that.

But, but what is an extreme risk? Like, what, what are we talking about?

So specifically in the RAISE Act, it sounds almost silly to describe how extreme it is.

Yeah.

We are talking about 100 people dying, killing 100 people, or $1 billion in damage, specifically via helping to build a chemical, biological, radiological, or nuclear weapon, or by committing a fully automated crime. Right. So this is not someone who's using the AI to commit a crime. It is You tell the AI to go do something totally legal, and it goes off on its own and commits a crime. And even if you meet that bar, you then have to prove that it was materially better than just using the internet and that that risk was in some way foreseeable.

I think one of the things that strikes me is that a lot of these— a lot of developers take these models and they apply them. The first place they apply them to is crypto trading. And arguably all of crypto is money laundering anyway. So it's all illegal anyway, right? We all know it's actually all illegal, but we have this, we all kind of agree not to look at it for now until the Dems come back into power, but it's all money laundering anyway, right? So it strikes me that all of these models are capable of and are going to do it, and developers are going to apply it to crypto trading of various kinds. And wash trading. And I used to be a financier, so I know all the tricks that you have. And in crypto, people apply all of them. It's like the 1920s, pre-SEC stock trading was like that in the United States. So I often wonder to what extent these bills do not recognize that that is the state of play, that all this illegal activity already occurs. widely everywhere. And it's just that prosecutors have difficulty prosecuting. The political winds are in the opposite direction for the moment, and which will swing at some point. And so I wonder how these regulations take into account the reality of the state of play on the field.

There's a lot there. So let me say, first of all, I don't think all crypto activity is money laundering. I just want to say that. I do support stronger regulation, and I think we should have some clarity But, but the key in the RAISE Act is if you were to— you as a user were to use a model to commit a crime, that is still a crime. Right. And that is still prosecutable. That is, we wouldn't need additional legislation for that.

Exactly.

The point of the legislation, and to be clear, does not change downstream liability, does not change what is a crime. Like all of those laws still apply. This is saying, like, what is the subset of extreme risks that are so bad that if they got out, if they happened, we wouldn't be satisfied to just deal with it later in the court process? Right. You want to actually prevent the bad thing from happening. I mean, you can think about in terms of, say, national security, which is different, different than law enforcement. Right. National security is about preventing something really bad from happening, even if it hasn't happened yet. And figuring out what are the ways it could go wrong and getting ahead of that. Where in law enforcement you're saying, okay, a crime has occurred, how do we deal with it? This is more on that national security side. There are things that are really bad. A bioweapon being created is exceptionally bad. Loss of control of an AI model is exceptionally bad. Those are the sorts of things we want planned for ahead of time and not dealing with later via lawsuits. And all the RAISE Act is saying is you as a developer, should have a meaningful plan that reduces the risk of those bad things happening.

Can we touch on the politics of this a little bit? Totally struck by the fact that, first of all, most of what you said there, and obviously devil's in the details and all that, but the general framework is very much an echo of the frameworks that the companies have put out themselves, for the most part, including OpenAI. And one would think that the— if you're a naive simpleton like myself, you would think that the tech industry would look at somebody who has a master's degree in CS and who even worked at Palantir for a few years and think like, hey, this is a guy we should be able to work with, right? And yet you've obviously come under some, I don't know, pressure, political attack ads, whatever, from, as I understand it, a super PAC backed by A16Z and Greg Brockman, and I guess I don't really understand why. I'm curious as to what you think their ultimate reason or objection is. But then I'm also really curious as to how is this playing out now in your race? Are they running attack ads on you that are on topic on AI, or are they running things that are off topic and kind of just trying to cause you trouble in general? And how are the voters responding to this? I kind of wonder if the AI industry might create a Streisand effect for themselves where they sort of end up doing their own cause more harm than good, because I think they should maybe look at some of the polling numbers on just like how popular AI is before they get too confident with, you know, some of their messaging. So a lot there. But what do you— what's your side of that story?

Yeah, you're spot on in your analysis. So I am running for Congress in a very crowded field. There's, I think, 10 declared candidates already in the middle of Manhattan. And this super PAC called Leading the Future was formed over the summer and was given $100 million funded by Andreessen Horowitz, by Greg Brockman, by Joe Lonsdale, by Ron Conway, and said they would go after basically anyone who was imposing regulation on AI. And there was actually funny politics over the summer where Trump initially opposed it and said, Don't support any Democrat that is like pro-AI. Like, you should just be a Republican. And so, but anyway, they came out with 2 people so far who they've named me, who they named public enemy number one. And then there's someone running for Congress, I think in Texas, a Republican who they really support. And his— the ad in support of the Republican barely mentions technology. It's just, this is a Trump-loving Republican and vote for him there. For me, they're actually very straightforward. And to be clear, these are smart political operatives who are doing it. I do not think they are dumb. If— but, but if you were just trying to bury someone, what you would do is you would wait until 3 weeks before the primary and you would put a bunch of negative ads on whatever the polls told you was most salient, not about the issue that you actually care about. And you do it right at the end so that disclosures would come out after the primary when it's too late for anyone to take that into account. What they did here is the opposite of that in every way. They announced early on who they were, that they were targeting me, that they were targeting me specifically about AI, and started putting ads out saying, don't vote for Alex because he'll regulate AI. And you're absolutely right. These ads, I think, are net positive for me, not only boosting my, my name ID in a crowded field, but saying that I want to regulate AI, which is a very popular position. So the question is, if they're smart, why are they doing it this way? And there's 2 clear answers to that. Uh, the first is that they want to discourage any other candidate from making AI part of their platform, right? It is clear that regulating AI is popular. People are figuring out what they're going to run on. They want to put a stake in the ground that they are coming after you if you ever want to regulate it. And the second reason is because the RAISE Act is currently before the governor. Their ad against me talked a little bit about me. It talked a lot about the RAISE Act, and it had a lot of images of Albany and said Albany is dysfunctional, Albany is dysfunctional. It was a very obvious message to the governor that they would come after her if she ended up signing a strong version of this bill. And so it's— it is often difficult to separate the politics from the policy. But in this case, they're being explicit in tying those two together.

Fascinating. And you think it is— I mean, I don't know if it's too early to have metrics on this, but you think it is actually benefiting you in your race?

So far. Now, listen, they originally said they were going to spend $2 million against me. Then I got a bunch of free press out of that. I got a surge in donations. I got a surge in volunteers. So they then upped it to they're going to spend at least $10 million against me. And I think if I keep at this pace, I could use up all $100 million of the super PAC and have to spare any other candidate from, from dealing with them. But, you know, it will be, it will be bad 3 weeks before the primary when they are dumping a bunch of negative ads. But so far, it's been very, very beneficial to me. That still means I'm going to need a lot of support from people so that when all that negative stuff is on the air even more, I can get my message out and fight back.

So I have a question for you, because unlike some of the other anti-AI candidates, I think you have been also clear that there are opportunities in AI. Totally. And I think this morning, Bernie Sanders put out a little bit of a spiel about how, you know, we should— they should block data centers and data center construction should be blocked. And this is something of the 1%, etc., etc., etc. So far, very, very negative reaction among the, uh, tech crew, both left-wing and right-wing. What do you think about that? How do you connect— how do you connect that, you know, a lot of the economic growth in the country right now is being generated by construction for AI data centers with— and, and a lot of the jobs are being generated, you know, and a lot of them are union jobs too, electricians.

Yeah.

Uh, you know, construction workers, etc. And how do you connect that with on the left wing of the party, at least this, you know, want to total shutdown on data centers?

I, I don't support a total moratorium on data centers, to be clear. Um, what Senator Sanders is reflecting though is that people feel like this is moving too fast and they feel like they don't have a say in how the technology develops and it's being decided by 5 people. And Americans deserve a democracy where we actually get to influence this. So, you know, what I would do on data centers, and to your point, I, I see the potential gains from AI. I use AI all the time right now, but I also, especially when it comes to medical research and curing diseases, I mean, my, my mom has multiple sclerosis and autoimmune diseases are some of the most difficult to understand. I'm really bullish on the ability of AI to have medical discoveries. It's just that these same pathways that allow you to have a medical discovery could allow you to build a bioweapon. And it's because I'm so bullish on the capabilities that I think there needs to be regulation. On data centers in particular, the American grid is so old and so in need of upgrades, and we need way more power as well. And we have stumbled into a situation where there seems to be near unlimited private capital willing to pay for a specific thing if we set up the incentives correctly. So what I would love to see the federal government do is instead of allowing states to be played off each other and instead of allowing long permitting times to have clear incentives set up where I can't get new renewable online for 4 years, that's way too long to benefit from a data center. But what I can do is go to an old coal plant that still has approval and fire it back up again, or I can go to a nuclear plant and say, hey, sell less to the grid and Give me some directly for a data center. And so that's what's happening right now. And that's why we're seeing even dirtier and dirtier power support this. I would love to see the federal government say we will streamline permitting, we'll make it much easier for you to connect if, A, you use a certain percentage renewable, right? It's tough to say 100% because you need baseloads and storage, but like a certain set renewable and you pay for the upgrade, you pay for the interconnect to the grid. So that, that doesn't get passed to ratepayers and ratepayers aren't subsidizing this. I think if you did that, you could have a huge benefit to our grid that helps everyone and not require a lot of federal money in order to do that. So I think there's a lot of creative things we could be doing here.

It's time to build, as they say. Dean is here. So just maybe one more question. I really am super curious about your answer on this one. With respect to Palantir, you worked at the company. I find AI scrambles all kind of politics. Everything is kind of topsy-turvy. But I'm struck by the sort of coded politics now seemingly being simultaneously super anti-China because they're a totalitarian state and we wouldn't want to live that way, and at the same time bringing this sort of mass surveillance tech to our own country. Right. And we've just had an announcement, for example, that people that want to come visit are apparently going to have to hand over their social media. And I assume it's going to be AI reading into that social media background to figure out who's desirable or not to visit here on vacation. I'm not sure exactly what we're supposed to be getting from that, but Palantir would potentially be the kind of company that the US government would engage to do it. So if you end up winning this election and going to Congress, How do you think you will engage with the fact that— and Palantir is always quick to say, well, we're only doing stuff that's legal. But Zvi, our first guest today, very often points out that so many things in society kind of rest on the assumption that certain things are hard to do. They're not actually scalable, they're costly. So it might be legal, but people aren't going to do it all the time. AI obviously challenges that assumption in a very fundamental way. The key question is, like, how would you engage with these sort of government contracting with a company like Palantir to spin up these sort of, in my view, kind of uncomfortably China-like surveillance operations? Where do you draw the line? Do you even trust Palantir? Do you trust Palantir, the company? Take that in any direction that you want, but I'm very curious as to what you would support and oppose in Congress.

It's a lot there. I'll try to be quick because I really want to hear from Dean, too. So you have a great next guest coming on. So I was at Palantir for 4 and a half years. I worked almost entirely on the federal civilian side, so I helped the Department of Veterans Affairs better staff their hospitals and ensure that veterans get the care that they need and deserve. I helped the Department of Justice tackle the opioid epidemic by finding doctors that were committing fraud against Medicare and Medicaid to do pill mill operations and just get people hooked on opioids. I worked with the CDC to better track epidemics. Like, I'm very proud of the work that I did while I was there. And also, I left in 2019 because they renewed a contract with ICE to allow the software to continue to be used for deportations, uh, and I didn't think that was the right thing for me to be working on. And so I took a stand there. The question on the social media coming from the airports and travel is like, That shouldn't be the policy of the United States. We should be quite clear, not only because of basic human rights and privacy rights, but also we're a country of immigrants. We're also a country that depends on people coming in and working here. I represent New York City, where we want people to come visit. We want people from all over there, and making that harder is such a self-defeating policy. So I think the broad policies of what we should and should not be doing and what people's rights are should be set. In Congress. Uh, it's just we've had a Congress that hasn't gotten anything done for far too long, and I think we need to send people there that are actually focused on doing things.

Indeed. And I think we may not have the same politics, but I do want to see more people who are more technically competent in Congress. So I wish you luck, Alex.

Thank you.

Thank you, Alex. Appreciate you being here.

Thanks for being here. Um, and everyone feel free to stay in touch @AlexBoris on, on Twitter and alexboris.nyc. Appreciate it.

We'll be following.

Hey, we'll continue our interview in a moment after a word from our sponsors.

Next up, we have Dean Ball. Let me get Dean on.

Hey, how's it going?

Hi, Dean. Welcome. Thanks for joining.

Of course. Thanks for having me. Good to be doing this.

So, Dean, you were the policy advisor on artificial intelligence. You were at the White House Office of Science and Technology. And then you left to— you said, I remember your tweet, you said you were a writer at heart and you decided to leave. So how are things going? What are you looking at right now?

Well, I mean, So the, the 4 months since I left government have been like among the most productive in my life. You know, there's just all these very cool opportunities to do interesting and I think positive things. And so the main thing I've been struggling with actually is like what to say no to. And like sort of like what the actual, like what, what is it that I want to accomplish here? And so, but I've been, I've loved being able to sort of speak openly again. Particularly because like, I think we, in, in my view, this year alone, we've had kind of 2 major moments that have redefined, at least for me, like how I, the level of utility that this technology has to me. And those were for me, like O3 was one, which actually came right as I entered, same week I entered government is when O3 came out. And then in the last couple of weeks, my view is that the sort of coding agents from the, the, at the frontier are like. kind of there, uh, and they can kind of do like basically anything you ask them to. And so, uh, so we're— it's just kind of been an incredible year in so many ways. And, um, I've been in— I've been enjoying being able to talk about it.

Can you talk a little bit— I know you heard at least the second half of our chat with Alex— what do you make of this dynamic where It violates my intuitions in so many ways to see a CS master's, worked at Palantir with, I would say, relatively light touch, kind of echoing the company's preparedness framework constructs in legislation, becoming the subject of attacks, which are— it sounded pretty honourable to me in the sense that they were at least talking about the issue that they really care about in the public.

Right.

Which is not always the case, right, in these sort of political operative type situations, but he thinks it's helping him. What is going on? Can you make sense of this?

Yeah, no, it's, uh, the politics of AI, politics of AI are in a very interesting state of sort of transition right now. I think, uh, the dice are really still in the air in terms of what the coalitions are going to be. Um, you know, it's, it's undoubtedly the case that like there's going to be, there's substantial public sentiment. on AI, negative sentiment, you know, thus far it's negative sentiment, but low salience, so low importance. But when you do ask people, so it's like, oftentimes it'll be like the 6th or 7th priority of voters, which basically means for like electoral purposes, it's not really a priority. In certain districts that won't be true, you know, all politics is local, blah, blah, blah. So I think the coalitions are in a state of flux right now. And I have said, you know, I'm sorry.

Just to interrupt you before we go into coalitions, what are the factions forming these coalitions?

Yeah. Well, so, I mean, I think even that is, is in a state of transition, but I think, you know, you have obviously like the traditional AI safety world. And then you might have like the sort of pro-AI, you know, industry heavy type of camp. Though it's not only industry, there's plenty of just like other people in it, myself included, by the way. And then you have what I would describe as this sort of emerging anti-AI.

Yeah.

Like truly, not like AI safety, but anti-AI. Those are different things. I think that's really important to understand. Like, I don't think of, of, you know, Assemblymember Borras as an anti-AI person. I think of him as a, as someone who is concerned about some aspects of AI safety. And I think there's pretty good reason for a lot of those things to be concerned. And so this anti-AI coalition is like, there's people on the left, you have Bernie Sanders saying we're going to ban the data centers. You have people on the right that are also doing this, saying these kinds of things.

And.

And I think then you have a bunch of people that are also worried about things like kids' safety, sort of more like consumer protection style harms. And so the real question is like, there's these 2 groups in the middle of, I'm worried about kids' safety and I'm more of a traditional AI safety person. And the question is like, which direction fundamentally are those people in the middle going to orient themselves towards? Are they going to decide that they're fundamentally anti-AI people? Or are they going to decide that they're like fundamentally, yeah, we want this technology, we want it to go well, and you think We think we need guardrails for that. It's a really interesting question.

Is there another faction? Because I often note that there have been several comments, we're not going to allow what happened to social media to happen to AI. And mainly not just about the safety aspects, but I think there was some loss of control that happened from, I think, public leaders to social media. They saw a loss of control.

Yeah.

Which, which I, I often say social media was actually the first proto-AI, and which, which caused— which already caused this kind of loss of control that the AI safety people were worried about, and that, you know, elected leaders could actually see it happen as they lost control of mass media and it became a social media which is mediated by, you know, these algorithms which delivered information to people. So to what extent do you think it's caused by this already kind of like fear of, you know, loss of control to AI.

Yeah, no, I mean, there definitely is some of that. And some of it I think is like, like this question of alignment, right? Of like, did the algorithms of social media, you know, did they, were they incentivized to do things that were actually good for society at large? Or was it sort of more locally optimal, but maybe globally quite profoundly suboptimal? And I think that there's like. Those are like important questions to ask. And we see the exact same questions present themselves in AI today, right? Like GPT-4o and Sycophancy is a good example of this. I think kind of a lot of the RLHF, heavily RLHF models have this sort of, especially like from the earlier part of this year, had a lot of these issues. Seems like they're getting better now, but yeah, like we see some of those same issues at the same time. It's very easy to. fight old battles. And so I see a lot of people that say that sort of thing. Really, they're viewing AI almost entirely as a consumer technology, like social media. And that causes, I think, a relatively myopic frame that in many ways totally misses the point. And the only other thing I would say though, is like, you're totally right that there's ways in which social media has challenged traditional institutions of media and tradition and the state in some ways, like really. Social media is like an authority destroyer. If you were like an old storied source of authority in society and credibility or whatever, social media has a tendency to erode that kind of thing. And there's ways in which that's good, but there's also ways in which that's like, you know, quite bad. And yeah, I do think there's some people who really understand this acutely. I would point to like Josh Hawley. I think Josh Hawley is like one of his true underlying concerns. He only talks about it sometimes, but I really think one of his foundational concerns, Senator Josh Hawley from Missouri.

Mm-hmm.

a Republican, you know, I think a big part of his concern is like this balance of power between basically corporate power on the internet and the government.

Indeed. I also note that for some of them, it's a question of wanting to tax the income stream, which they see that the, uh, the big firms have a very enormous income stream right now. And I think, for example, you can see it in kind of Marsha Blackburn in Tennessee. You can see a large portion of the regulation that they want to put forward is to protect artists in Nashville who they're afraid are going to lose their livelihoods because of corporate power in California.

For sure. I mean, of course, you know, whenever people talk about, oh, we have to protect creators from AI, I think to myself, like, well, in some ways, like, all 3 of us are like creators, you know, of different kinds of stuff on the internet. And like, I, speaking for myself, I can just personally say, like, I don't really think any of that stuff particularly protects me. Feels like it's more about protecting holders of very large IP portfolios.

Mm-hmm.

So it's like, you know, the, the interests of Disney and the News Corporation and similar large media conglomerates, I don't think their interests, like, we shouldn't factor in their interests to nothing, right? It's not like, you know, but like, I'm, my only point is like, let's not pretend this is some populist cause where we're like defending the copyright of like, we're pretending the intellectual property rights of like the proverbial musician on the street, you know, with a dream and a guitar, right? I don't really think that's what's happening here. We're protecting Taylor Swift, who's like a billionaire and that's fine. But come on.

I guess to protect myself, I would like to propose a ban on NotebookLM. So that everybody has to listen to real human voices.

Yes.

Okay, here's a question I really want to get your take on. And obviously there's a certain, you know, he who must always be named that I think of as kind of a great example of this. An idiosyncratic person you've called a political genius, and I think it's hard to argue with that, but who has made some kind of perplexing moves recently in terms of an EO, which I think seems to be not popular among his base, right? If I understand those politics correctly, like, My sense is that people who voted for the president did not do it so that the president would take away states' rights to regulate technology. On the other hand, we're now going to sell H-200s to China. So you can comment on those particular details of those perhaps surprising actions if you want to. But then zooming out, if I think about what really matters in history, I would usually think of myself as more of a structuralist thinker that's like, what are the really kind of core technology drivers or the core economic drivers? And then, of course, you've got people sort of squabbling for position on top of those more fundamental dynamics. But I don't tend to think of the individual leaders being super key forces that shape history until right now. I'm kind of like, oh my God, maybe this is a sort of moment in history where the great man of history theory Could perhaps have a lot more to say for it because so much is up in the air that maybe these like individual decisions, like selling H-200s to China, really could become like historical hinge points. How do you think about that?

So I think you are, I think you're spot on that like right now we are in a moment. I, I mean, I think that Donald Trump, say what you want about him, I do think he will probably go down as being like a great man of history. A person who like really changed the orientation of this country, changed sort of the lexicon of, of our statecraft and our civic life. Like, and I think, you know, if we're being honest, like with anyone like that, where you're living through the impact of what they're doing, I think you have to be honest that history will be the judge of all this. I think that the, like, in terms of the specifics, maybe like, so I think the president is one thing on the right that I think a lot of people don't understand, like a lot of the people who would be opposed to something like preemption. Or at least would say they are, is like, they have this belief that like the president has been corrupted by the tech industry, that like it's the tech industry pulling the strings here. And the president couldn't possibly disagree with them on the core of this issue. And it's like, no, the president does. Like, the president cares himself about the issue of preemption. It's very clear from the way he talks about it. It's very clear from how consistent he's been on this issue. He personally cares about it. He personally wants to do this. And, you know, he's ultimately a leader. He's going to lead the party where, you know, where he thinks it needs to go, what he thinks is in the best interest of the country. And I think he's right that preemption is in the best interest of the country. In terms of the H-200 thing, uh, I mean, so much has been said about that. I think it's like, we'll see what China even does. Like, are they even going to take the chips? How many are actually going to move? It's so hard to exactly know. I, you know, I personally, like, I think we need to be balancing here between like, we want to give China, we want China to not like completely abandon. We don't want to, we want to make sure that we are selling stuff that is better than the best stuff they can produce such that like we do maintain some ecosystem advantages that are real, right? Like there's Chinese engineers who help optimize, you know, who, who help write CUDA kernels and do other sorts of optimizations to the basic infrastructure of machine learning.

Mm-hmm.

And like, That is a big part of how NVIDIA achieves performance is through the contributions of this ecosystem. And so in many ways, like those things have benefits, not just for NVIDIA, but for American developers of all sorts who are building within the NVIDIA ecosystem. That's an important thing. It's an, you know, I mean, other just sort of U.S. ecosystems in general, right? Is it the most important thing we need to be thinking about in the context of this competition? No, it's one of many important things. So you have to balance that with the national security. Security issues. And, uh, yeah, yeah, we'll see. Kind of also China has to do their own balancing act, so we'll see where they, where they, you know, we'll see what they actually do. It's—

hey, we'll continue our interview in a moment after a word from our sponsors. Fascinating. On the China point, I was really struck recently. I ran your, uh, discussion with Max Tegmark as a cross-post on the Cognitive Revolution. Thank you. And in scrolling through the names of people who signed on to the superintelligence ban statement that you guys were debating, 2 names jumped out at me above all others. One was Steve Bannon.

Yep.

The other was the CEO of Zhipu AI or Z.ai, which is one of maybe the top 5 or 6 Chinese AI companies. They're the makers of GLM. The latest one is GLM 4.6. So that's kind of odd. What do you make of it? I don't know. Is there any— I mean, it seems like we are maybe in a sort of Nixon goes to China proto moment here, perhaps. Is there any— do you have any optimism for that?

I think this is where the whole great man of history versus structural forces thing is really going to come to a head, because my view of the situation, and I've written this publicly, is that we are in a a state of structural decoupling from China that it doesn't actually matter. In other words, if I'm right, it doesn't matter who the president is unless, unless the U.S. government starts to take way more control over the economy. Like unless we become more centrally planned as a, as a, as an economy. But if we don't, and we continue to have a market economy, which means that private capital allocators make choices about how to allocate capital, they're going to look at the policy landscape and it's going to be like, well, look, I can't rely on China for rare earths. Because they've just expressed a willingness to like use this entire supply chain to, to derail my business. So I can't do, like, I'm the fiduciary of a company. I cannot do that. You know, we've already seen US companies do that and a lot of them are doing much more of it right now. So we're in this sort of state of structural decoupling. China is exporting all this stuff, which does lower prices, but it also competes with domestic industries, not just here, but everywhere.

Mm-hmm.

And so there's a lot of people that are having problems with this all over the world and are like, what are we going to do about this? Like, it seems like what China wants to do is extract raw resources from my country and transform them in its own country to things that it sells back to me. And that's basically how China views the future of the world, which is like not a very cooperative or positive vision for the vast majority of people that are not in China. And I don't even think it's that good of a vision for the people in China, honestly.

Mm-hmm.

I think it's a good, I think it's a good vision for the Chinese government. I think we're in this structural, you know, decoupling. The president thinks differently. And I think, I mean, my read on the president is that like, he is genuinely interested. He doesn't want to be the person who started a 100-year civilizational grand competition. He wants to be the person who established a framework for peaceable relations between our countries and therefore really for the world. He wants world, I mean, he really does genuinely want world peace. And maybe he can do it. You know, I'm not, I'm not going to bet against POTUS. He has surprised me way too much for me to bet against him.

One thing you tweeted recently, Dean, that I thought was quite interesting and quite true is that for the purposes of making sense of what's going on in AI, a lot of the best information is private and is sort of circulated, you know, person to person at the proverbial San Francisco parties, what have you. Could you share in sufficiently abstracted terms so as to not cause yourself or your sources problems that would help people who are not as plugged into the whisper networks have a better sense of what to expect in the next year?

What to expect in the next year? Well, I mean, so I think very broadly, there's a lot of things in AI right now that are just like, there's a lot of stuff that's quiet because it's sort of politically incorrect, almost like I felt like yesterday I made the pronouncement that I feel like Claude Opus 4.5 and Claude Code is like just AGI, basically, according to like some definitions at least. And like, that's the kind of thing that is like fairly politically incorrect to say. And like, might've been like, a lot of people might say it like behind closed doors, but you don't, because you don't want to be, you know, there's so much career risk of like, oh, am I going to be the guy who came out and said this thing is AGI? AGI is an unserious concept, blah, blah, blah, blah, blah. I think that one of the things I would guess is that the conversation will in some ways become a little bit more candid, I hope, over the next year, just because some of the realities are going to be harder to deny. And so there's a lot of things right now that like on both sides, there's a lot of like foundational assumptions that both sides have stuck to throughout the year that have sort of undergirded their rhetoric and points of view. And I sort of wonder whether that will— It's not that they won't, it's not that they'll like be totally, you know, neutral or whatever. It's more that I think that the assumptions will start to change and therefore—

The load-bearing assumptions are starting to go away.

Yeah. I think the aperture of like stuff you can say and policies you can contemplate will widen. I basically think the Overton window will widen. And, you know, I think other stuff too, like the models are going to be really good and they'll probably be autonomous cyber attacks and all that. But yeah.

Yeah. The first, um, drop-in knowledge workers expected Q2, uh, 2026, from what I understand, as well. Amazing. Well, Dean, thank you very much for joining. I appreciate your, uh, participation in our live experiment today. And, um, it's always great to get your take on these big picture questions. You're a unique thinker and we appreciate it.

Next up, we have Peter Wilderford. He is the co-founder of Metaculus. He is a policy strategist. He is the globally top-ranked top 20 forecaster. This is going to be exciting.

Hey, yeah, thanks for having me. Sorry to take you away from Dean, but happy to be here. By the way, I'm not the co-founder of Metaculus. I wish. I'm on the board of Metaculus. I'm an avid Metaculus participant, but yes, I work at the Institute for AI Policy and Strategy.

You guys are— Since we were just talking about chips, you have been an opponent of selling chips to China. I think that's fair to say. What's your take on the latest development?

Yeah, I mean, I think my take is that, like, obviously there's something to be said about wanting to undermine demand for Huawei chips, but I think the mistake that people make is that they assume that China is like a capitalist country and will respond like To supply and demand the way that other countries do, and that's just like clearly not the case. Like China, the Chinese government will make sure that Huawei has unlimited demand for their products. Right now, Huawei can only supply like one to four percent of Chinese demand, and the Chinese government is going to make sure that there are companies that take that demand by by law by soft requirement. And so then the question is like well what about the remaining ninety six percent that's like not being able to be filled by Huawei? Well, I think in that case, like that's where Nvidia is kind of a useful idiot. Like they're kind of filling in that like missing 96%, and China is going to be like, thank you for helping turbocharge our AI industry. But then the moment Huawei can take in more and more demand, the government's going to be pushing out Nvidia and going to be promoting Huawei, putting really, really firm thumbs on the scale there. And that's like that's what China's done in every other industry. Like Tesla's come to China thinking they can sell a ton of electric vehicles. They got pushed out for BYD. Lots of other companies think they're going to, like Apple thought they would be selling a lot of devices in China, got pushed out also for Huawei. Yeah. Like the Chinese government, like Uber, I think also got pushed out. Like, yeah, just like this is how it works in every industry. And so I think we need to have more of an understanding of how supply and demand actually works in China and like how, how these chips like are actually being used or, or not used to, yeah, like what is the, like, how is Huawei being undermined? Like, I really don't think they're being undermined at all. I think China is kind of, they're having their cake and eating it too by, by boosting Huawei domestically and also leapfrogging off of NVIDIA chips. Yeah. So I feel quite strongly about that.

So what would you say if the chips issue was put in as a part of a strategy to kind of stress the Chinese government into overspending. So it's not that the US policymakers really believe in AGI or ASI, but it is kind of a red flag to a bull, kind of causing the Chinese government to overcommit on construction, overcommit on chips, At the same time, they're going to have to match SpaceX on launches. They have a real estate bubble, which has just burst, and they're trying to address that. So they've got all of these things going on at the same time, a lot of needs for capital, a lot of capital burn. BYD and the other EV companies are now all negative capital, and the government is having this anti-involution kind of campaign to prevent people from Producing at massive losses. So basically, it's it's similar to the strategy that the U.S. used against against the USSR, which was to start off the strategic defense initiative, stress the stress the USSR on spending. At the same time, oil prices went down. The USSR then had a collapse. So in the same way, this is maybe even policymakers do not believe in AGI or ASI, but it is a very useful kind of way to encourage spending in China and overspending and overcommitment. On these strategies.

Yeah, I mean, with all due respect, I do think that's kind of a bit of a galaxy-brained take. Like it seems to be really relying on second-order effects and ignoring like the the premacy of the first-order effects. If you believe, I guess what I do about AGI, ASI, like I think a lot of this like big spending is very rational. China might actually be underspending relative to the advantage that AI can create for a country when it becomes kind of like super powerful, as I'm expecting. I think, yeah, like the US is— if the US is going to win in AGI, it's going to be largely by outspending China, by having deeper capital markets. I think that's always been the US's strength. And I do think the Chinese government, the Chinese economy is under strain. I think the China can only prop up the government so much, and I think that they had a lot of pain from zero COVID, which I think was a tremendously ill-advised policy, and I think shows that the Chinese government isn't omni-competent. But I think if like if the U.S. just wants to like trick China into spending, I mean like why not just like race to Mars or something? Like there's like far more, there's like far more safe things we could be doing than giving China everything they need to like power their military, power their economy and like, and like, it's clear, like China's weaponizing their AI against us. They're like cyber attacking us relentlessly. Like they're, they're not playing fair with, with the technology that they're using with us. And they're also using AI to do tremendous human rights abuses domestically. And like, I don't think it makes sense for us to be powering that.

So is that the core of your objection? I feel like. One of the things that always gets kind of glossed over, often gets glossed over in these sort of why should we or shouldn't we sell chips to China debates, is like, what is the actual threat model? My sense is like, they have enough production to power their drones. They probably have enough production to power their surveillance stack. And even though at one point in time, right, I once ran a deep research report on the history of justifications for the chip ban. And it sort of started with like, well, we'll deny them the military applications. But then it was like, well, we can't probably do that, but we can at least deny them the ability to make frontier models. And I was like, well, now we can't really do that. And the latest version that I've heard from Lennart Heim, who's kind of been a big advocate for this policy, is like, well, at least we'll have a lot more AI agents than them. So we'll be a lot more productive economically. And they won't be. And when I hear that, I'm kind of like, I don't know, man, that sounds not really super consistent with my values. I'd like to see the Chinese public enjoy the benefits of AI agents in their day-to-day life, as I hope to enjoy them in my life. It doesn't seem to me like we can really do much about their military applications or their surveillance tech or even their cyberattacks. Those are going to happen. I also note that my life goes on pretty well today without too much interference from actual cyberattack damages. So the other, of course, answer that people give is like, well, if they get this, then there will be like 5+ more companies in the AGI race, and that's just going to make the whole competitive dynamic worse, and we'll have even more of a rapid race to the bottom. And that resonates with me a bit more, to be honest. But then I'm also kind of like, well, maybe we should regulate our own companies a little bit if we really want to do something about that. So I guess, All that to say, like, there's kind of a menu there you can take from the menu or go off menu. What, what is the core threat model that you really worry about when China gets chips?

Yeah, I mean, I want to be really clear, like, it definitely should not and it cannot be the policy of the United States government that we want to make China poor. And like, we should not be trying to imply that to China at all. Um, I think China deserves economic success as much as any country does, as much as the United States. And I completely agree. Like, I want the Chinese people and even the Chinese government to be benefiting from AI technology, even benefiting from American AI technology. I think when it comes to the chips in particular, though, I think the threat model is like kind of like once you give chips, you can't really take them back. I think it also is just kind of— it's really underrated. I think the amount of the current Chinese AI ecosystem, DeepSeek, Quinn, etc., that are being built on American chips. It was just reported by the Information that DeepSeek is smuggling American chips. I think if America really wanted to and like really could crack down on chip smuggling, crack down on chip like sales, like there, I think China would be much further behind the frontier, especially with enough time for those policies to work. And I think like the core threat model is that I think AI will be kind of like able to, yeah, like you said, drop in remote workers earlier, like kind of drop in military analysts, drop in like military commanders. I think a lot of like military AI will be just like a tremendous strategic asset. And I think we want to be really careful about like, I mean, China has so much civil-military fusion. I think unfortunately it's like very difficult to be giving China these equipment that then can be militarized. And like, yeah, if AGI does really take off, and if there is some like, God help us, some like US-China great power war, like we're not going to be wanting China to have all those like drop-in military strategists. Like we want, we'll want the American military to have that. And so I would say like we should rent chips to China rather than sell them. We should make sure China can benefit from American AI rather than have the materials to compete with American AI. And we need to be careful, I think, about the military implications, kind of like how I think the United States should sell a lot of things to China, but we shouldn't be selling our fighter jets or our missiles to China.

Mm-hmm.

Like, I guess that's like obviously more of like a clear example, a little bit inflammatory deliberately, but kind of like the way I see chips and the way I think military will eventually be tremendously superpowered by AI. I think that like is a non-trivial portion of the Chinese AI ecosystem and eventually their military capability. And I think we obviously are doing fine now, but like if we do need to be prepared for like some sort of Taiwan scenario, sometime, some type of great power war scenario, we, and we shouldn't be suffocating China in the quest to do that, but we should be thoughtful about like, what are we or are we not selling? And like, what are those implications?

What is the difference between, let's say, China buying the chips and setting up data centers onshore and ByteDance buying the chips in Singapore and deploying them in an Oracle data center in Malaysia across the border? What are the major— because, you know, and as you imply in the civil-military fusion, I'm sure ByteDance does some work for the government, right?

Yeah. I think they definitely do. I mean, I think the biggest difference with renting chips as opposed to buying Is that if there were some sort of war, you could always turn the chips off. If they're like kind of more on Malaysian shores, yeah, we just have like so much ability. Like if they're controlled by Oracle, we just have so much more ability to shut them off if if we think that they're being misused. And and then I think also there's like a bit of a dynamic in China too, where the military will be tremendously disincentivized to use Malaysian data centers. Because they don't want the United States like spying on their military like workloads. But commercially, I would imagine Chinese companies would be happy to use Malaysian Oracle-run chips. So like that, I think that actually is a great way to benefit the Chinese people economically without benefiting Chinese militarily. Interesting. Yeah.

Maybe maybe rent don't maybe just segue to Nathan go ahead.

No, I was just saying, I like the rent, don't sell paradigm a lot.

Yeah. Yeah. I think rent, don't sell is my favorite way of trying to square the 2 things of like, yeah, I mean, I am definitely like a China hawk, but like, again, I really don't want the Chinese people to be poor. This is not like an economic war against China. This is just like sensible military policy. And I think permitting renting, but not selling them the chips, I think is the best way to square that. Um, I know other people disagree with me on the renting thing. Um, and I'm happy to have that debate with them as well. I guess this sort of feels like, uh, maybe a more moderate position in that case.

Indeed. I'm going to add this to the stage, which is the, uh, the famous Arc AGI-1 leaderboard with the 390 times improvement in one year. I think that you had a tweet that this was surprising to you, or?

Yeah, I mean, I do think it is surprising. So one of my favorite things about this evaluation is like, I mean, I feel like every eval we ever see like emphasizes like, oh my gosh, like AI is improving so much in like ability to get stuff done and capability. And like, that's super cool. But I think what goes a bit unnoticed is just like how much cheaper it's getting to do things as well. Like, so the, the old capabilities you used to be able to do just barely can now be done by open source models and maybe even models that you can run on your laptop and soon models you can run on your phone. And I think that has a lot of implications as well in terms of what we're going to be able to do with AI. And I'm very excited about that. And so I think just kind of seeing like, wow, not only is there tremendous increase in capability, but tremendous increase in affordability. And like, you kind of like get to multiply that, like affordability times capability. And I think that just really shows an awful lot of opportunity, especially as that presumably will keep continuing.

Is this one of the trends that you foresaw in your 2025 forecast? Did this surprise you?

Yeah. I mean, I must admit that I didn't like look at this trend in particular, but I guess I do. Yeah, I find myself sometimes a bit surprised by, yeah, just like how quickly it is that you can make, take existing capabilities and like just make them a lot cheaper, even sometimes only over a matter of months.

I looked back at our respective AI 2025 predictions and not surprisingly, yours seemed to hold up a little bit better than mine. I overestimated the benchmark scores. by a bit, pretty much across the board. You were a little more accurate. I think I was a little closer to the revenue number, though, which was the one number that really blew out relative to the sort of aggregate community forecast. So this is kind of an interesting tension where the benchmarks haven't gone quite as far as I expected, but the revenue has really ramped. And then when you juxtapose that against the fact that the same capabilities are getting dramatically cheaper, and yet we are still kind of telling ourselves all that so far would suggest that, well, we must be seeing just like infinite use on these sort of mundane tasks. But then we also have the narrative that, well, we haven't really seen much economic impact yet. So how do we square those 3 things? Obviously, progress has been pretty significant. But a little less than I expected. Revenue growth has been huge, even as cost has come down. And yet we still are like, where's the impact? Like, what do you make of that sort of very seemingly confusing constellation of facts?

Yeah, that is confusing. I must admit, I did a really— I made a pretty big mistake when trying to forecast revenue for this year, which is basically at the end of the year, companies, OpenAI, Anthropic, et cetera, all just like reported here's how much revenue we expect to make over the next year. And I thought like, oh, well, those are the company's official projections. Like surely they know much more about revenue than I do. Like surely you can just like sum them up and that's total revenue. And like, why would you try to do something else? And I think like, yeah, one thing I learned this year was like, I guess when companies put together their revenue projections, They have to use like really standard economic models and math, and they're not allowed to like account for the craziness of AI progress because that's just like unprecedented. And you're not allowed to like put unprecedented stuff in your economic models or you look unserious. And so, yeah, in retrospect, I think that was like a really dumb prediction on my part. I definitely should have known that like, oh, the revenue will obviously be a lot higher because I was expecting AI progress that wasn't being priced in by like the standard economic models, which I think are primarily backward looking. Yeah, I think it seemed like revenue this year was like kind of really explosive off of coding models in particular, like Claude Code, Cursor, Windsurf, et cetera. Like that really sort of was a breakout use case. I was expecting, like one thing I was a little Kind of, you said like overestimating capabilities a bit. I was expecting more economic value just from like other sorts of computer use agents besides coding. That kind of didn't really take off this year. It wasn't really the year of the agent as much as like I or others might have said. But like, I think AI just became tremendously good at coding in particular, as like Dean Ball was talking about. And that just ended up being a very economically valuable Use case. Like, I'm always surprised. Like, I was, I was literally a professional software engineer, data scientist for 5 years. I was coding every single day, hours on end. And even these days, like, Claude code so much faster, so much better than me. I basically like rarely write code myself anymore. Just like always would rather like have Claude code spit it out for me, at least for the first draft. So I was very impressed by that.

Yeah. I think just to add to that point, the thing that I always point out is that the CFO of the company has to have some sense of reality when dealing with investors. And so the researchers do not. So there's this gap between the researchers and the management, and the CFO often externally facing has a certain number of things that they are allowed to say and a certain number of things that they are not allowed to say. One of those things is declaring a 10x on revenue in any year is not something that anyone will take you seriously, because they will ask you, what is the product that you have? Who are the customers that you have signed? How does this money come in? Just to take a step back, Claude Code at the beginning of this year, They launched it late last year. It had pickup, but it didn't have this kind of 10x pickup. It was growing very strongly, but to 10x in one year is amazing. It's fantastic. I also wonder to what extent do you think they can 10x again next year? Because again, projections for next year are kind of like 2x to 3x for each of these firms. Do you think they're going to exceed again?

Yeah, I mean, I would think yes. I mean, I admit there is a certain level of 0 to 1 where like people weren't using AI at all and now they are using AI and like maybe some of that can't be recaptured. And so that obviously would like slow growth down a bit, but like, I mean, I think kind of like the true, like game-changing AI use cases, like still aren't really here. Like if. If companies actually can land something that is like akin to like a drop-in remote worker or like drop-in management consultant or something, like companies pay like hundreds of thousands of dollars a year for that sort of thing. Yeah, we're not talking about like $20 a month for your Claude Code subscription, but like $20,000 a month for your like remote worker subscription. And Yeah. And so again, I think it really sort of depends on, like, yeah, can this, like, train of increased capability continue? And, like, yeah, like, I really think computer use agents could be enormously economically valuable if they could actually be delivered in a sufficiently reliable way that, like, meets the promises that people like myself have been hyping up a lot over the past few years.

What are the sources that you are watching? You mentioned, obviously, like, the company's official revenue projections. We've got, you know, obviously Twitter where people are sounding off all the time. I do find that honestly to be still my primary source of up-to-date information. But then there's all these prediction markets as well, you know, some of which are more kind of like Metaculus is a little bit more geared toward like the true, you know, forecaster's forecaster. And then others are more kind of betting markets. Interestingly, it seems like recently we've seen sort of a surge of insider activity on some of the betting markets, which has been a really interesting phenomenon. But what are the sort of, you know, SIGINT sources that you are tuning into to make sure you are as informed as possible that you would recommend other people pay more attention to as well?

Yeah, I mean, I think you're totally right. Twitter's my go-to as well. Also, yeah, pretty happy with Zvi's like aggregation of Twitter on his Substack. I think a lot of other Substacks are pretty excellent as well. Yeah, like Tim Lee's, like, Understanding AI or Nathan Lambert's Substack, I definitely like a lot. I'm also, yeah, a big fan of following SemiAnalysis, following Epoch AI, following Meter. I think kind of these organizations that have like a reputation for not like hyping stuff up, not vague posting, but like kind of really getting into the details of like, where is AI on these evaluations? Like, how are things? trending. And more importantly, like, what do these evaluations actually mean? And like, what are they covering and what are they not? Something I've really liked seeing from Epoch AI, for example, is like just seeing like real professional mathematicians like comment on how useful they're finding AI math assistants. And like, yeah, like I think Daniel Litt, for example, on Twitter, like every— a lot of times when a new AI model comes out, He puts it through its paces with his professional mathematics work. And I mean, these are AI systems that can win mathematics challenges and they definitely seem to be approaching some level of value, but they still seem to— and they're doing so well on this frontier math benchmark, but they still seem to be lacking in their ability to deliver for professional mathematicians. But I think as that changes, we'll be seeing from these in-person taste testers. I've also been kind of taste testing the models myself, especially with more writing or policy analysis tasks that I think are just not really well covered by benchmarks. Yes, I think getting a holistic picture seems important. I also like that METER looks into reliability as well, because I think it can be one thing to do well on some hard problems in a benchmark, but another thing to be reliably delivering in a professional world. For example, if I only succeeded at 80% of my policy analysis, I don't think I would be keeping my job very long. And so, yeah, it's just like the jaggedness of these models, they can succeed a lot, but then just mess up on something really stupid makes it a bit harder for them to be a drop-in worker right now as opposed to a working assistant, which I use them For in an assistant way.

Do you assign like a probability of being correct or wrong, like handicap some of these organizations and statistics more than others?

Yeah. I mean, I think so. Like, I mean, I've always been, yeah, a bit frustrated, I think, especially by OpenAI where they like do all these like vague posting, hyping their models, like posting Death Star on Twitter. And talking about how their model is going to be like, or like, I think Elon Musk, like every single model he's ever built, he's like, oh, this is the one that's going to be AGI. And you're just like, yeah, come on. Like, stop, stop doing that. Like, it's just really unhelpful and like also needlessly stress inducing for those of us trying to follow this kind of stuff. I feel like now you can just sort of tune it out. But yeah, I think like it was just like adding more noise than signal. In the past. Yeah. So, I mean, usually when models come out, I just advise everyone to like ignore the news for the first 24 hours. Like every tester's like, oh, I got early access to Model X and it like was the best model ever. And it like did all my homework and it like built me a new website and it like, like cured cancer or whatever. And it's just like, well, the only reason those people tend to get early access is because they're gonna like write glowing things about every model. no matter what. So like, I really like, again, Epoch and Meter, organizations that like have more of a track record of not— of like just kind of, yeah, calling it as it is as opposed to hyping up the model or saying that the model is stupid when it's not.

As we look ahead to 2026, I think it's hard to— when you get into forecasting, which I've done a little bit, I— a little lore about me, I participated in one of the original Good Judgment Project forecasting tournaments like 15 years ago. I did okay, but not as well as you. I actually did pretty well, but still not as well as you.

Nice.

One of the things that I've learned though in doing this is it is very much anchored in questions where you can get a number out in a lot of cases, right? It tends to lend itself toward in-paradigm changes. We have a benchmark, current number is X. What's it going to be at the end of the year? How do you think about things that are a little bit more paradigm shifting? Like, if I were to ask you, what are the odds in 2026 that we get a new paradigm on the order of the chain of thought reasoning paradigm that took us from sort of language models 1.0 to 2.0, let's say, A 3.0 could be a bunch of different things. It could be a continual learning breakthrough. It could be something else. It could be embodiment in some way, shape, or form. How do you think about trying to get a handle on those things which are not like a well-defined scale or move, but rather something that's kind of qualitatively different than what we're currently dealing with?

Yeah, I think that's definitely an astute observation. I think with forecasting in particular, there's so much fuss put on the task of taking an existing forecasting question that has well-defined resolution criteria and put a number on it. And that kind of activity, I think, is an activity I'm very good at and does have a lot of value to people. But it's only a small part of the overall forecasting value chain. I think 2 things that are really dismissed is First, what questions are you asking in the first place? What are you actually posing to people? What are you allocating your attention to? And then secondly is once you have this number, this prediction, what do you actually do about it? How do you make decisions different than you otherwise would? And yeah, so I think the full forecasting value chain is way more than just putting a number on stuff. It's also asking the right questions and doing the right things with the information. And yeah, I think a lot of times, a lot of AI forecasting can be really stuck in assuming current paradigm continues and current paradigm is the only paradigm. When I think looking back throughout history, we have, I think, a decent amount of evidence that we get new paradigms, honestly, even every 5 years, twice per decade or so. So it stands to reason any particular year might have a at least a 10% chance of having some new paradigm developed that does qualitatively change model performance. I think one of the interesting things also with the reasoning models is we all assumed pre-training was the only kind of training. I think previously it was just called training and you would just have more and more FLoP for models and you were expecting more and more returns from that. But it kind of felt like pre-training, I guess, has been burning out a bit. But it started burning out a bit right at the same time reasoning models started taking off. And so you almost continued this linear AI progress just through a second era where the first thing kind of stopped working, but this new thing picked up just in time to continue it. And so I think you could actually get maybe surprisingly linear progress despite a changing guard of paradigm. But yeah, yeah, I think definitely it's like sort of, yeah, difficult black swan kind of events to like think about like, oh, will 2026 actually see some form of continuous online learning? Will it see some form of like, oh, like the chain of thought's no longer in English now. It's like thinking in like raw neural pattern and that improves its ability to like represent concepts and it's just like so much better now, or like, will there be some new data efficiency algorithm where rather than requiring 100,000 examples, it only requires 10 or 15 like a human and it can learn so much faster that way. I think we really haven't scraped out what's possible. So yeah, I think it still does widen your uncertainty definitely when thinking about next year or even the year after that.

What do you think the end of 2026 looks like? The most likely case in your mind? It can be anything that you envision, but what do you feel— what do you think it feels like at the end of 2026?

Yeah, so I think by the end of 2026, I think you'll— so like, yeah, I mean, one evaluation I've been tracking a lot that I like a lot is like METER's evaluation where they evaluate AI on their ability to do software tasks and then they compare Roughly, like, how long does it take human professionals to do that task? And right now AI is achieving like 50% reliability on tasks that take human professionals about 2 hours. And so like, that's enough to get like reasonable value out of like Claude Code or something, but still like, the human is like very much involved. But I could see at the end of 2026, I think it could be possible. That you could be having AI that could do like a day or 2 of human labor. So like, it would just be like significantly more autonomous, significantly more able to just like run with things, do all the trial and error itself without having human input every like 20 minutes, like Claude Code currently does. And I think that really could transform, yeah, like transform kind of how we relate to AI and like what AI is capable of. I think one other interesting finding from METER that they looked into was like also it's not just like, it's not just software that is like increasing gradually in capabilities, but like kind of almost a lot of tasks, like a lot of tasks you can measure, whether it be computer use or like economic analysis or like legal performance or like self-driving cars or anything, kind of have like similar capability increases over time, just starting from much lower bases. So right now, maybe an AI can only do a 10-second computer task and then it gets confused or whatever. I would imagine by the end of the next year, I would expect AI to be using computers at least a human level. And so I think that would really transform again. What AI can do, like if, if AI can sort of do, like, I mean, the computer right now, the internet's just kind of the gateway for like a lot of economic activity. And if you, if you could start delegating that to AI and just have even like some semblance of reliability and like maybe even like multiple trials, like it tries 5 times to do the thing and messes up 4 times, but that like 5th time it like knows it succeeded and then it like reports back and maybe it's like horribly inefficient. Compared to a human doing it, but it's like so much cheaper and it can also run while you're asleep or run while you're doing something else. Like, I think that still would be pretty valuable. And I think that will like, yeah, again, transform what we could be getting from AI. Yeah. So I know I said, I kind of said 2025 would be the year of the agent and that sort of didn't really take off. But there has been a lot of improvement in computer use. And so I would think that like 2026 might be the year of the agent in some sense. I'm happy to just repeat that prediction again and see if it sticks again. this upcoming year.

What do you think will be the political ramifications of that? Because you've already got, like, I think Senator Fetterman was— he wants, if you're going to have self-driving trucks, he wants a Teamsters representative on board. Bernie Sanders this morning with, you know, moratorium and data center buildouts. What do you think as you get to the year of the agent? What do you think the politics of this looks like at the end of the year?

Yeah, I mean, I think the politics will be heating up, um, definitely. I think we've already been seeing that. I think 2026 will also, like, you'll be seeing Waymos and self-driving cars, um, way more commonplace, hopefully. I could see, like, if there was some, like, AI agent, even if it did, like, fairly rudimentary things, like, if, if it was, like, affordable and, like, fairly universally used, like, I think that could be another ChatGPT moment, like kind of, yeah, in 2023, like the first, like actually usable chatbot with a good UI. The trends were all there. Like all of us studying AI, like weren't that surprised by ChatGPT, but like, because it had such a slick user interface, like it created like a transformation in politics. And then all of a sudden we were having like international summits on AI and stuff when previously it wasn't the talk of. tech policy at all. I could see like, oh crap, like the AI is not a chatbot anymore. It's actually doing things in the real world. It's like getting my Uber Eats and it's like booking my plane tickets and it's like able to do these actual practical things. Like, I think you could more viscerally feel how that leads to unemployment. Maybe it actually does like tick up unemployment in like a very small amount, but like in a large amount to be noticeable. And then I think, yeah, you're heading into the midterms and like we're already seeing just this past year, there's been a lot of populist anti-AI sentiment. Like the data centers are, there's a claim that they're like using all the water. There's a claim that they're running up electricity prices. Like people are already on the watch for affordability. And I think they like don't like big tech. It's easy to blame the data centers for your problems. It's easy to blame big tech for your problems and maybe you don't feel like you can really do anything about OpenAI or Facebook or something, but you definitely can do something about that data center in your neighborhood. And so like it is kind of a way for individuals to feel like they have some power over what's going on. And I'm expecting, yeah, politicians to continue to use that. I definitely think, yeah, like AI, it's already been kind of a big part of tech policy this past year. I think it'll be an even bigger Part of tech policy in 2026. Unfortunately, I don't feel like we're going to really land on any actual solutions that get passed. I think kind of most midterm years tend to be fairly acrimonious and not really solutions-oriented. And I think that's super unfortunate. And I want to work on that. I want to make sure that we're getting good use cases like Waymos, while of course not like running up way too much. risk, um, like from AI run amok, like AI with no guardrails whatsoever, like, like a lot of big tech seems to want. Um, but like finding somewhere in between, um, and like harnessing that, that anti-AI sentiment towards like more productive solutions, um, seems like a really important thing to do this upcoming year.

Last one for me. Do you want to venture a year when I'll get a domestic service robot that will come into my house and Do my laundry and cook my meals and make my coffee in the morning?

Yeah, great question. I mean, I think that like robotics has always been on the disappointing side, and I think there's also just like not really good evaluations for it. But I kind of have a sense that like, like once robotics starts working, it will just like really start working. And so like we may just see kind of a lot of robotics progress kind of feel like it comes out of nowhere. Sort of like, I think how self-driving cars like weren't really economically viable for the longest time. Like, I remember like even way back in 2015, I thought I might have a driverless car pretty soon. And that was like just wildly off. I wasn't a good forecaster back then. I was just a college kid. But like now we're finally seeing it happen about 10 years later. So like, I mean, I would venture maybe, yeah, by like the end of the 2030s, you have sufficient AI progress and sufficient follow-on robotics progress and sufficient reliability that, and also just like sufficient cost reduction that it's just like cheaper to hire the robot butler to come instead of the human butler. Obviously, I guess that would be like tremendous for individual consumers who then can like outsource a lot of their chores. I certainly would like that. Of course, that's like obviously very difficult for the American worker, um, that would normally work that job. So yeah, there are definitely really important trade-offs there, and that's going to be a really important policy conversation for sure.

Yeah, it feels to me like we're maybe 4 years behind in robotics compared to the chatbot. Like we got ChatGPT late 2022. We're going to start to see the first humanoids shipped to consumer. homes next year. It seems like they're probably not going to be great. They might have a sort of, yeah, GPT-3.5 level reliability. But if it keeps pace, then in a, you know, in a 2030, you can imagine getting AIs along the, you know, getting robots that are sort of performing similarly as a like Claude 4 or 5 Opus perhaps.

Yeah.

And that would be, you know, well, not necessarily perfect and able to do every last thing I might want in my home, you know, that would definitely be a product I would be keen to, uh, sign up for, especially if it could— yeah, yeah, the character of, uh, Claude.

Yeah, that would be super fun. Yeah, I mean, I think we haven't really seen the ChatGPT moment for robotics yet, but like, yeah, I mean, the ChatGPT moment was only 2, 3 years before the Claude 4.5 Opus moment. Um, so yeah, I mean, definitely once you start hitting on something, it can be like very quick, um, until it's like very useful.

Yeah, I feel like the physical manufacturing, servos, etc., are actually in better shape than the AI used to control these things. Like, there's a lot more.

Yeah, yeah, I don't know. But one thing I did learn recently was I think like robots only have like 10,000 tactile sensors on their hands, which like Sounds like an awful lot of sensors, and that is definitely a lot to cram into one robot hand. But I think humans have like only almost a million different touch points on their hand. And so yeah, there's just like still so much scaling to be done to meet human performance. Yeah, it really does speak to that Moore's Paradox that like, yeah, doing the dishes is just like so much more difficult than like doing a math competition when it comes to AI.

Yeah, it's incredible.

That might be a great place to leave it unless you have any other bold predictions you want to give us for 2026 or beyond.

Well, I'm hoping— I'm predicting a successful live show and hoping you'll do another one next year. And I'm wishing everyone, yeah, a Merry Christmas or a merry happy holidays of their choosing if you're not of the Christmas persuasion. Um, indeed. Yeah, looking forward to it. Thanks for having me on.

Happy holidays. Appreciate it, Peter. Happy holidays.

But, and wow, what a show. What a marathon.

I'm not even sure I can properly step back to reflect on it just yet, but yeah, yeah, it's certainly fun. And I do think we, up until the end there where we kind of let the time with Peter go along, We at least stayed on time.

That was by choice because we'd been pressed for time the entire series. Every speaker you want to continue and then you end up having to cut them off. It's a tough one. I found Boris interesting. He's very charismatic. It reminds me that politicians are much more charismatic than tech people. Very charismatic. I tried to draw him out on the crypto stuff. He didn't bite. So I think that was interesting for me, interesting moment. I found, I think Dean was much more optimistic, I'd say. I found him like very optimistic. So that was good. That was it. That was a good differentiation from Zvi who started off quite, I think, quite negative. And overall, it also struck me that One of the things that struck me was how many of the developments that we were talking about had really only happened in the last couple of months, like the GPT-5.2, the 390x cost improvement. That was just a few weeks ago. Nested learning was not that long ago. It's really— when you talk to people about biotech, they talk about things that happened a year ago or 18 months ago. And you can really see how fast things are diffusing, at least from the research to the community, but perhaps not quite into the economy yet. And you could feel that. I could feel that everyone— I think everyone on the show expected more and still expects more for the next year. Which I think is a little bit different from the market right now. The market thinks AI bubble, this is it. So the market has kind of flattened out expectations for the year. It's very interesting to hear the insider views, which are quite different from what the market believes.

When it comes to the market, one thing I always wrestle with is, and I think actually Noah Smith had a pretty good write-up of this recently, Where he was like, the tech is for real. You're not going to get a sort of bubble burst when it comes to what AI can do. Obviously, it can already do a lot, and it'll continue to do more. But that doesn't necessarily mean that there will be durable high profit margins for the companies that makes the financing of all this make sense in the fullness of time. So I wonder how you think about that, because I'm kind of like pretty confident that we're going to get to some kind of— as Dean already, you know, Tyler Cowen called AGI with O3. Dean is kind of soft calling it with Claude 4.5 Opus.

Yeah.

I, you know, if we don't want to call it yet, I'm still confident we're going to get to something that will qualify. But I'm not sure that, you know, because I— and this is one thing I wanted to get to with Logan, but we didn't quite have time. It seems like switching costs are rising, but they're still not that big. And part of the reason people maybe don't switch as much as they otherwise could is just because all the models are pretty cheap. We're all benefiting from the competition. Nobody wants to be dramatically undercut, so they're all pricing pretty low. But it does feel like we could see sort of progress continue, but also price wars continue to the point where maybe the economics of it do kind of go haywire, even as the capabilities, you know, progress, maybe not quite as fast as I predicted, but, you know, not, not necessarily too dissimilar. What do you think of that?

So one thing that strikes me is that the useful models may get smaller over time. Like Gemini 3 Flash, for example, is, is a, is a breakthrough in efficiency, obviously, like it's as It's on par with 2.5 Pro, but it's so fast. It's obviously a much more economic model to serve. And so that makes me think, okay, if you have a model that can do a software engineer's job today, but it requires a Blackwell chip, and in 12 months you have a model that can do a software engineer's job, but it requires an H100 chip, Meaning you go back in the chip, you get so much more efficient that you're using an older chip. Do you still make the same economic value from the software engineer? Because it's the same work that's being done, like on the B100 or the H100 in 12 months. And so that speaks to me that, you know, this is what Jensen said, the H100s are still being used because as things get better, you are able to deploy more advanced, smaller models on older chips. And the economic value that those models are providing is still there. And that really confuses— that I think really confuses a lot of the assumptions that people have. It's not that you need more, like, better chips for more capability, but you can also serve the same capability with older and smaller chips. And that's something that we've seen even, for example, ArcAGI, the tiny recursion model was a 7 billion parameter model. So much smaller. And if the economic task was ArcAGI, I'm sure in 12 months you'd have a 7 billion parameter model, which was at 80% scoring equivalent to a 5.2 Pro. So over time, a job is a job. That software engineer's job is still being done. It's being done on an older, smaller chip. That economic value is being created. If you have a lot of software engineers, the demand starts to— you get demand, you get oversupply, obviously. But in a world where we have too few software engineers and where software engineering creates a lot of value, and over time, this idiot index of our brains using 20 watts to produce a software engineer and these models using 1 megawatt hour to produce a software engineer, and that we're closing that index, right? That idiot index is closing over time. And I feel like that means that you get more economic activity on older chips over time. And I think that is something that will become more apparent. You can see it in the booking numbers for some of these firms for the older chips, but I think it'll become more apparent by the end of next year because I think there'll be more Claude code type software engineers out there creating more code. And that, I think, might be where you start to see, okay, this is real economic activity. Even if you spend on these data centers right now, they are actually going to be useful 6 years from now, because 6 years from now, we will have so much better models, smaller models that we can still deploy on these chips, and we can still get economic growth out of them. I don't know. That's what I feel. What do you think?

So does that bottom line out to, you think the financing can be sustained because these things will have a long serviceable life, basically?

Yeah, I think so. I think the way, for example, a Meta thinks is that if I get superintelligence, great. If not, we're going to generate lots of slop AI. And the slop AI, they're definitely going to do personalized automated advertising. It's going to come. You're gonna open up your Facebook and you may have someone who looks very similar to your wife or your kid pitching you on a product. It's gonna get real, right? And video gen takes a lot and a lot of GPUs, and the economic benefits are gonna be there for advertising. It's clear personalized advertising will come. And that is like a fallback position for these companies, for Meta, for Amazon. Worst case scenario, do slop ads. You have the economic benefits there. And that's how they think. They build out to support that they're not going to go bankrupt and they can still build these things. Worst case, they go into junk bond territory. They live a few years with junk bond type credit ratings. But they're confident in their cash flows because at this point, A lot of the economic activity in the US is now concentrated on these $10 trillion companies that are really reshaping the global economy as we speak. It does worry me that it's a one-way bet though. It's a one-way bet for the entire US economy right now. There's no diversification anymore. It's all in on this. China is not doing that. China is diversifying into multiple other things. They're spending money on a lot of other stuff. It does worry me a little bit that it is a one-way bet, but you have people who are willing to gamble at the top, so they're gambling.

Yeah, that's maybe a great cliffhanger note to leave us on. Thanks for helping put this together. I really enjoyed getting to talk to 9 different leading experts, and I think there's obviously ample opportunity To do it more. I just made a little list of things that we didn't really cover. Mechanistic interpretability, opening up the black box, didn't touch on that at all. Alignment and control techniques, we basically didn't touch on at all. AI for science, we kind of alluded to a few times, but there are a functionally infinite number of experts at the intersection of AI and all these different sciences that we could be talking about.

That was—

Media gen, we didn't either.

There are these people doing reinforcement learning with science back and forth. That could be, uh, that's another thing that we, we didn't explore. Uh, Nathan, thank you. Uh, this was, this was awesome. And, um, yeah, let's, uh, let's see if we can do it again. Let's see, let's see what the responses are online. I'm very curious.

So let us know what feedback you have if you made it this far. Uh, bless you, and, um, we'll definitely be very interested in your feedback. Thanks, Prakash. This has been fun.

Thanks, Nathan. Cheers. Bye-bye.

Bye for now. If you're finding value in the show, we'd appreciate it if you'd take a moment to share it with friends, post online, write a review on Apple Podcasts or Spotify, or just leave us a comment on YouTube. Of course, we always welcome your feedback, guest and topic suggestions, and sponsorship inquiries, either via our website, cognitiverevolution.ai, or by DMing me on your favorite social network. The Cognitive Revolution is part of the Turpentine Network, a network of podcasts, which is now part of A16Z, where experts talk technology, business, economics, geopolitics, culture, and more. We're produced by AI Podcasting. If you're looking for podcast production help for everything from the moment you stop recording to the moment your audience starts listening, check them out and see my endorsement at aipodcast.ing. And thank you to everyone who listens for being part of the Cognitive Revolution.