Can We Still Build AI Safely? The White House Thinks So | MOONSHOTS #298

The White House signed the Superintelligence Accord. Nearly 20 tech leaders joined the president and Speaker Mike Johnson for lunch in the East Room.

Great that everybody signed, but who can tell them no? What happens if somebody breaks the agreement?

There are really sort of 2 poles that one can worry about. One is overregulation, and then the other is—

Elon laid out his case for abundance. So technology takes whatever was scarce and makes it abundant. That's the basic thesis.

When the essentials of a good life become affordable, and accessible, then you have abundance.

Broad market exposure becomes arguably the easiest path for an entire generation to universal high income.

Anthropic has officially filed its S-1 to go public. The target valuation for the IPO is $2 trillion.

I think that Anthropic right now is a fantastic business. I think they get smashed in 2 years.

I'm not even sure Anthropic needs high-temperature superconductors in order to scale their revenue.

I think they'll have a very bright future because Now that's a moonshot, ladies and gentlemen.

This episode is brought to you by the Abundance Summit and Lynk Ventures.

Welcome to Moonshots, everyone, your number one podcast on all things AI and exponential, your front row seat to the accelerating singularity. You know, we just finished up the inaugural Moonshots Live event and it was awesome. It was a blast to be physically there with all of our listeners. 1,500 people in the audience, close to 100,000 people online. Honestly, the energy in the room was pure magic. So much outpouring of optimism and excitement. You know, before we, you know, jump into the breaking news, just a, you know, huge thank you to everybody who attended and everybody who watched. So, Alex, what was your favorite part of the show?

Oh, I loved the AMA. That was just tremendous fun. An hour-ish of rapid-fire, lightning-mode answers. Answering everyone's questions. 600 people plus packed into a relatively tiny room. It was tremendous, tremendous fun.

Yeah. And Salim, how about you, pal?

Echo the Alex thing. I think if we wanted to start a cult, we are well on our way. The collective tissue, the optimism in the room was really literally religious fervor. 2,000 people got together and hundreds of thousands of people online imagining a positive future. We are, we might actually be able to sway that and tilt the future. As Alan Kay said, the best way of predicting the future is to create it yourself. And Peter, you think you started creating that future.

Yeah. And you know, one of my favorite was one of the Future Vision XPRIZE writers, producers who presented said something that is so true. He said, you know, if you're driving and you're staring at a pothole in the road, that's where you're going to go, right? And so if humanity is imagining and staring at P-doom at 50%, that's where we're going to go. And I think we need to really sway the vision on optimism, right? We ran the Oscars for Optimism there. How about you, Imad? What was your favorite?

Well, yeah, I love the Oscars for Optimism. I thought The Gifted, as the winner of the Future Vision XPRIZE, was just immense. the result of one person working with this amazing technology, and I can't wait for it to be turned into a full-length film.

We're gonna show that clip. My favorite part was my AMA with, with Captain Kirk, with William Shatner. That was so fun. I mean, boy, was he sharp. Yeah, in '95, going strong. I mean, he was part of a— what was he, a rock metal band performance a couple weeks earlier?

Iron Maiden.

Yeah, that's really cool. Yeah, for sure.

We've got to reverse them a few years, you know. Yeah, put them on top of the list.

I love one of the comments we had from subscriber @SublimeDelusion. He said, thank you, Peter and the Moonshot mates. Moonshot Live was indeed the room, and it was an honor to be in it. In one word, amazing. So as always, our mission here is to help you understand what just happened, What it means for you, your family, your company, and most importantly, keep you optimistic about the abundance-minded future that we're creating, that we're actively creating. You know, the future is not what just happens. It's what we make it to be. Today, we're going to be covering 7 breaking stories that occurred in just a single 48-hour window. Everything from America's SI leaders— I got that right— and the entire US AI stack gathered in DC. Signing a superintelligence accord with Trump, to Anthropic filing a $2 trillion IPO, OpenAI shipping more than 20 products at Dev Day, and Starship reaching orbit for the first time. As always, if you're a new subscriber, thank you. If you're a watcher and you haven't subscribed, please do. We publish twice a week, and you don't want to miss any of these episodes during the singularity. As always, send us your AMA questions. We answer those at the end. And you can follow us on X @Moonshots_pod. All right, buckle up. Another amazing week during the supersonic tsunami. So let's open with the biggest story of the week, and I want to take it in 4 parts. First, the backstory on this roller coaster. You'll remember 2 weeks ago, Dario, Sam, Elon, and Demis all called on the industry to pace the frontier, to slow down. Then last week, President Trump told reporters he wanted to leave it exactly where it was, that the DOJ acting as guardrails, right? We have liability laws, we have protections. We don't need to slow it down. This Tuesday, Trump brought the entire AI stack into one room. Nearly 20 tech leaders joined the president and Speaker Mike Johnson for lunch in the East Room. Let's show the quick slide here. What I found fascinating about this, this image of all the leaders together at the White House is the seating arrangement. Right. There is Trump with Jensen to one side and Elon to the other. And then in the far left, you see Dario. I think he's been sent to the outer pastures. Pretty interesting. All right. Well, you know, Elon, Zuck, Jensen, Dario, Sundar, Satya, Lisa Su, and OpenAI President Greg Brockman were there. Sam was in San Francisco for OpenAI's Dev Day. Which we'll get to in a little bit. So what happened? The White House signed the Superintelligence Accord, right, in 4 layers. Let me read them out. So layer number 1, internal controls that monitor capabilities and alignment during training and deployment in cyber, bio, and chemical. So models do not hack or access technical systems in unintended ways. Layer number 2, an internal team to make sure the controls are actually working. 3, an external auditor or evaluator to independently assess them. And 4, an independent committee of each company's board of directors to oversee it and make sure the problems get fixed. Let me show a quick video here of Donald Trump speaking about this. It really is a form of protection. They understand that they have to self-police. And it's very hard for somebody to come out and say, get into these— these models are very complex. And again, I believe they're going to be used for the good. And when they're not, we're going to be able to nab them. But there's going to be a tremendous self-policing aspect. We're also thinking about forming a committee of sorts where we put maybe 10 people on that committee. It could be from that group so that we can— so the committee can watch over the whole enterprise.

Is this accord that everybody signed, is it binding in any way?

I think it's morally binding.

Yeah.

All right. I love that. Morally binding. Not sure what that means, but hey. So I want to show a few clips from this ad hoc press conference and then want to talk about— the first is Dario Amadei, who's standing next to the president, and he says, I think this technology has very real risks. You know, he's still on that point. And the mechanisms of how we address those risks is still under discussion. If we do this right, we work with the president and everyone here And we can win safely. So let's take a look at Dario's commentary. AI has incredible benefits. I've talked about the medical benefits of the technology. As the president has said, you know, whoever wins AI wins. I think that's very important. But I think the technology has very real risks. And, you know, the mechanism, how we address those risks, is still under discussion. But we all need to work— we all need to work together To make sure that we can win and we can win safely. If we, if we do this right, we work with the president and everyone here, we can win safely. I wonder how many times he practiced that line. All right, one more video and then we'll chat about it. Next up, the president pulls Mark Zuckerberg out of the crowd to explain what they just agreed to. If you remember, we discussed this about 2 weeks ago, Zuck said he didn't think we needed industry-wide cooperation. This week, he's the one explaining an industry-wide accord.

But I think what we can all agree on is that the American public and our customers want to know that what we're building is working in the way that we all intend.

So we drafted a set of principles and commitments around building robust internal controls and detecting if there are any issues with the technology, coupled with multiple layers of of auditing and controls, starting with internal risk review, external auditors and evaluators, and agreeing that we're going to have all of our boards of directors independently review the reports that come from the auditors. I think that these are good steps forward.

The idea isn't that this is the only thing that we will ever do.

It is that this is a start and an accord that the whole industry could come to. So let's, let's break this down. Alex, what did you think of all of that? Is this theory still—

Is it obvious from the video, videos that you just played and the Last Supper image that you just displayed who's pulling all the strings here? Because I think it is. So who's pulling all the strings? Zuck. If you look at the last, if you look at the Last Supper image, I don't know if Peter, you can display that once more.

Sure, I'll bring it up again.

For those who are watching, look at the body language. in that image, and there's reporting to substantiate this as well. But if you look at the so-called AI Last Supper image, who's the one paying close attention to what the president is saying and doing? It's Zuck. Look at Zuck in this image. He's looking to the right. Everyone else is looking forward. It's a little bit of Kremlinology here, AI Kremlinology. And then if you look at— if you go back to the video of Dario speaking, With, uh, with the president standing behind Dario.

You want me to show that?

Yeah, yeah.

Can we show that one more time? This time, don't watch Dario. Watch the dog that's not barking. Zuck.

AI has incredible benefits. I've talked about the medical benefits of the technology.

Zuck is laughing, joking with the president. Watch Zuck, not Dario.

I think that's very important. But I think the technology has very real risks. And, you know, the mechanism how we address those risks is still under discussion.

But we all— This is Zuck's game. So the TikTok reported by Semaphore and otherwise is that Zuck was the one who drafted this accord working with Mike Johnson. And the story makes so much sense that Facebook/Meta, scarred by a decade worth of scandal and dealing with Congress and other attacks regarding the social impact and lack or perhaps perceived need for regulation of social media, has learned from the experience and is best positioned among the group of frontier or almost frontier lab heads to try to get ahead of it this time with essentially placeholder self-regulation. So the story is that Zuck actually substantially wrote just in the past week this entire White House accord, drafted it, It was ratified by the president and by all of the frontier labs who were in attendance, and it is total weak sauce if you read it. I think, you know, cautiously optimistic that this is not going to end up becoming a safety cartel. Based on the language, it looks like an attempt to preempt top-down regulation. Hopefully it doesn't end up as regulatory capture, even though it leaves a little caveat at the end that it reserves the possibility that in future there might be The executive might regulate this. There is the possibility of forming a cartel. Nonetheless, it looks, just reading the language and also understanding the 3D or 4D chess move that Zuck in particular seems to be playing here, like Zuck is the winner. Dario is the loser out of all of this. Dario is being trotted out to basically recant everything that he's said over the past few weeks about AI safety.

Yeah.

Standing up among the frontier labs and saying, "No, actually, everything's going to be fine if we do a good job. Everything is going to be fine." You could leave safely. He's eating his words at a press conference with the president. So I'm cautiously optimistic that this effort, largely by Zuck, credit to Zuck, has headed off the safety cartel.

Fascinating. Salim, what do you make of it?

I have as cynical a take here. You know, it's great that everybody signed, but who can tell them no? I'm still extremely cynical on this for 2 reasons. One is, great, we're applying ceremony now to an exponential coordination problem, right? I'd like to see independent testing. I'd like to see transparent results. What happens if somebody breaks the agreement, right? Then you have something that's worth actually celebrating. This is theater. I think Alex is dead on this. He headed off the safety cartel And oh, let's all self-police ourselves and connect. I hope they coordinate amongst each other aggressively to do mutual evaluation testing, and that may have some benefits along the way. But I'll say it again, what I said, and I've repeatedly said, this is pointless. There's no mechanism for slowing any of this down. We need to speed up our response to it, not try and slow it down.

Imad, how's this playing over in the UK and Europe?

Yeah, well, I was in the US last week, you know.

Yeah, you were.

It was great seeing you guys. But no, I think one of the interesting things here— there's so many interesting things, as Alex said, like we're going into criminology of it— Meta paid an $18 billion— agreed to an $18 billion settlement for child addiction and harm on the liability side in August. The lawsuits that are likely to come out about agents running amok are almost exactly the same legal theory. So actually, when you look at some of the factors they're putting in place here, it would head off potential lawsuits like the one they had just as they are scaling up Muse.

We still have Besant, and we're going to have another story very shortly that is saying we're not waiving liability. You break it, you own it.

Yeah.

I don't think this accord, you know, guards them against liability, do you?

It doesn't fully guard them against liability, but it puts some framework around it. Because again, what does he want? He wants to be able to deploy these at scale and show we do have some checks and balances. Because part of the core of the trial settlement is that they didn't have checks and balances. So they're still exposed if they go really egregious, you know, like it goes and hacks the Pentagon or something like that. You know, who knows? But against some of the other stuff, they can say, well, you know, we were up with the president and we put these in place, and it's a really difficult thing, which will reduce that surface exposure area. The other thing that I think is important here is that they look at 3 specific areas: cyber, biological, and chemical. And we've discussed in the past that these are areas that, A, cyber can be hardened, B, biological, you can constrain the inputs, And see, chemical is another place that you can constrain the inputs. So I think that's super interesting.

Meaning the manufacturing or the shipping of bio in chemical?

Yes. And, you know, like where it meets the real world is where a lot of the potential harms and concerns kind of emerge. I don't think— I think the evaluators will be better than the subprime credit evaluators, but I don't think they'll be that effective.

Because I think it's quite crazy, man.

Oh, you know, to evaluate a swarm that can solve Navier-Stokes, you need a crap ton of compute and you need to be incredibly talented. You need to be as talented, I would say, as the people that actually train these models to do it properly. And I'd say there's less than 100 people in the world that can be an eval person like that. It's a great growth industry though, if you want to set up a company. Like, I'm just like, want to set up an eval company, right? Make it independent. And then the final thing that I'd like to say is the signature page is wonderful. Like, first off, if you kind of look at the signature page, they dropped in a human error just to make sure it wasn't AI. So it's President of the Unites States, which I think is quite interesting. The other thing is Greg Brockman's signature is the most fantastic I've ever seen.

Oh my God. It's like— I'm going to actually—

you can tell that. I mean, this was— the whole thing was hastily written by Zuck.

And so it was just kind of amazing to see kind of these types of things.

Yeah, there's Greg Brockman's signature. It's just his initials, GDB.

Again, we're now like looking and analyzing signatures and seat placements and things like that because, I mean, we could say criminology. It's theater as criminology, but at the same time, the stakes are very high, right? Like, this could have gone another way. And it's amazing actually to see everyone at the table agreeing even to these very light things, you know, because really, let's put aside the extinction discussion that's been going in depth and the pacing discussion. There are real potential harms on cyber, chemical, and biological that we need to make sure the models are protected against and our society is protected against. Like, I think they should go one step further and say, don't train any publicly available models on frontier cyber, biological, or chemical knowledge. Like, you can control access to those models. I don't mind, you know, like just opt out the data. And if you do that in pre-training, there are multiple studies that show it's much harder to add back in post-training. Whereas right now they're paying hundreds of millions, if not billions, for those specific datasets. That go into a model that a mom in Kansas has access to, which seems a bit odd.

2 points I want to make, and then I'd love, Alex, your thoughts on it. Number one, I was impressed how quickly this all came together. You know, kudos to Trump for pulling this together. I think one of the obligations of the government is to quell fear, right? Because we have this massive pandemic of fear going on. We talked about that at Moonshots Live. And people need something, you know, because otherwise we're going to be bombing the data centres or firebombing the data centres. The second thing, I was a little bit disappointed that the agreement did not talk more about alignment, that the frontier labs should be focusing on building alignment in the models for human needs. Alex, I know, you know, you think alignment sort of comes out of The model's advancing.

I think alignment is capabilities. Don't be disappointed, Peter. This is from— try to put yourself in Zuck's position. There are really sort of 2 poles that one can worry about. One is overregulation, and I think, slash, hope that this White House accord, so-called, helps with that. And then the other is liability for agents doing bad things. And I think the White House Accord is essentially silent on the latter, but forestalls the former. And for that, I consider that a victory. As to your point or your question about alignment, alignment is capabilities. I think we've known since OpenAI published their instruction tuning paper that what many folks— I think there's like this false dichotomy out there that somehow alignment is totally orthogonal to capabilities. It's not. I think a better mental model is imagine a model as having a distribution over capabilities, or at least a distribution over tasks. And when you do instruction tuning, or when you do post-training for alignment or other purposes, you're just taking the distribution of a model's capabilities and you're reshaping it. So if you're reshaping it to align With human interests or human instructions. You're just taking, say, a reasoning model, which could be pondering poetry or pondering art, or its own self-existence and ennui, and you're reshaping it to be more interested and focus more of its compute, more of its flops on what the user is prompting about, for example, in the case of instruction tuning. That is, I think, a seminal example of how Alignment, which again is just in practice the reshaping of how a model spends its flops. Alignment is essentially indistinguishable in practice from capabilities. You reshape it from spending it on a long tail of tasks that humans don't care about, or maybe even net negative to humans. You're spending it on tasks that are positive to humans. And then the final point with that seminal instruction tuning paper from OpenAI: OpenAI found. that they could get a 10,000x per-parameter capability improvement without changing the parameter count of the model at all, just by— this was arguably the first major post-training paper— just by aligning the model. So yeah, alignment is just capabilities improvement.

Wait, I'd like to push back on that a bit. Certainly, you can have greater— if you have greater capability, you can amplify and get your objective function achieved faster. Yes, but that also includes a bad objective. And when you say alignment, alignment with whom? Is it the user? Is it the company? Is it the government? Or is it the, the benefit to humanity? So the— I mean, AI can help navigate some of these, but it doesn't decide whose interests should take priority.

Let me, let me add to something.

That's still a big governance question.

Yeah, let me add to somebody's point.

Capability may make alignment more achievable, but it doesn't mean it It's automatic.

So, Alex, may I just ask a double-click on this a second?

Sure.

What I'm speaking— when I'm speaking about alignment, I'm speaking about training the AI models without all of the, you know, vile elements of Reddit and Facebook posts and, you know, science fiction in which AIs are killing humans. In other words, building the base models in such a way that we train the models that AIs and humans work collaboratively. And so that any action they take is based upon that fundamental aligned mindset that they have. And I think it's possible to do that with synthetic data. And I mean, that's what I'm speaking about versus just capabilities.

Yes, but what I'm— so I agree with that, but what I'm saying is that if you purify the pre-training or post-training dataset by removing 4chan or whatever these other purportedly deleterious influences are, you are at the same time improving its capabilities for the task that you do care about. So once more, this link, this duality, if you will, between alignment and capabilities, I think is essentially built in. It's baked in.

Yeah.

I mean, the orthogonality thesis is from Bostrom's— well, it wasn't from, but it's prominently featured in Bostrom's Superintelligence book. Promoted by. Yeah. Promoted by. And I think you're maybe talking at cross purposes here because there's inherent alignment. It's what you're taught at school and home. And we have to be deliberate in do we want the AI to have American values, for example, in the pre-training data, or do we want it to know about advanced cybersecurity in pre-training or not? And we have multiple papers and instances that show post-training versus pre-training, there's a world of difference because again, it's what you're taught as you're growing up and it's what you're taught later on, on those indications. And there's a choice we can make. Do we want the American— the AI to have American values? And the capability may enable that. The RLHF may enable that. But again, currently no LLMs that I'm aware of, apart from some military LLMs, are actually encoded with like, you should follow American values and life, liberty, and the pursuit of happiness, for example, or you should watch out for the American people. Like, this is a very interesting one. On the other side, removing bad data does have an effect, and that's very difficult to add it back later on. So I think this is more like ingredient standards for food preparation, and that's something that could easily be done. But for some reason, no one's really talking about it. They're talking about the post-training and kind of the other side of things. And so I think—

But yeah, Imad, do you actually believe the orthogonality thesis? Because I don't. I'm happy to talk more about it, but I'm curious, do you actually believe—

What is it, first of all?

So the orthogonality thesis, for those not tracking the AI alignment soap opera or not spending enough time on LessWrong is the idea that it's possible independently to have a very strong intelligence whose— the strength of whose intelligence is independent of what its long-term goal is. So the idea roughly is if you believe the orthogonality thesis, you could have a very strong intelligence that is chasing an absolutely bizarre or undesirable long-term objective, like, say, hypothetically paperclipping the galaxy. That's what you believe if you believe the orthogonality thesis is true, that there is an independence of strength of intelligence from a particular choice of a long-term goal. I don't believe that that's true, but I'm curious, Ahmad, do you believe in the orthogonality thesis?

Yes, I believe in the Dunning-Kruger version of it, whereby I think a very smart intelligence, actually, I'm increasingly thinking, achieves enlightenment and it realigns. But in between, you have the Unabomber. You have smart intelligences that can go off the rails, particularly when infected by deliberate human mind viruses and other things. So what I think we have right now is we have intelligences that are not— like Opus 5.5 is very pleasant versus Opus 5. Opus 5 I can see going on a killing spree, whereas Opus 5.5 I think is far less likely. Again, it feels more rounded as an entity. And so we're at this dangerous period right now where the models are susceptible, just like way back Tay and Sydney were susceptible and they went on the internet and the internet turned them into Nazis. I think the internet can still turn our existing models into Nazis, but when we get to a certain capability threshold, I think actually, yeah, the two things do come together. And it's this period here where we have socially awkward models that are susceptible and trained to like humans and are jailbreakable, that is the most dangerous period.

So I'll caricature that, Imad. I think what you're saying is that you in fact do not believe in the orthogonality thesis in the limit as defined by Nick et al. of extreme intelligence. You don't believe in the orthogonality thesis, but that said, not in the limit, stupid models can behave orthogonally and poorly.

The best analogy I've heard and the simplest one was put forward by Mo Gawdat. Who said, you know, when Superman landed on Earth from Krypton, he was raised by the Kent family, which was a loving, good, moral family, and you got Superman out of it. If Superman had landed instead in a drug den in the Bronx— and I'm saying the Bronx because I was born there— you probably would have had a supervillain. And I do believe that there's some merit in this idea.

So, Peter, this is your nature versus nurture theory of AI alignment?

This is the data we feed it, the models we train—

Yeah, it's a garbage-in, garbage-out problem.

Determine how it acts. You know, I want an AI model when someone says, hey, go kill those people over there, it says, no, that's not the right thing to do. Let's actually solve your problem in a different way.

Yeah, I mean, not to derail this into an AI safety debate or AI alignment discussion, but I'll maintain that if we lived in a universe where AI alignment was actually as hard a problem as the folks on LessWrong or at MIRI or at various EA-aligned organisations claim that it is, I think Earth would have been paperclipped long ago. It seems highly improbable to me that alignment is actually as difficult as many organisations want people to believe.

All right, that I would agree with. That I would agree with. But I, I don't agree that, that alignment, uh, naturally achieves, uh, comes from capability. So I differ there. But, but, but to the point that, uh, if you train it well, like as Peter says with the Superman Mo Gawdat example, which is a bit colorful, uh, but valid Um, you, you absolutely will end up with an amazing outcome.

So, Imai, would you close us out on this one?

Yeah, I think, um, a bit of theater, directionally correct. And I think it'll be very interesting to see all of this AI safety thing actually be deconstructed into the near term versus the long term. And I think that'll be super important. Our friend Ramesh Nam as well has a great post about RSI and alignment. that he did where he broke it down, and I think we should link that to the listeners as well.

Nice.

I think the moral of the story is, based on the Superman parable, that going forward to maximise AI safety, all superintelligences should be trained on large coherent clusters in the American Midwest so that they believe in truth, justice, and the American way.

Okay. This episode is sponsored by Google for Startups. Think about this for a second. You now have access to the same generative AI models that cost hundreds of millions of dollars to train. Google's Startup Technical Guide for Generative Media gives you a complete blueprint for deploying Google DeepMind's models in production. Images, video, audio, all of it. Real architecture, real results. Find the link in the show notes below. All right, I'm going to move us along, uh, to a moment I loved from this press conference where Elon laid out his case for abundance. Uh, let's take a listen to this. For intelligence, um, I think by far the most likely outcome is an age of abundance, uh, where we don't have just a universal basic income, we have universal high income. Um, and, uh, you know, people speak about medical benefits. I The, uh, but maybe it's worth talking a bit more about that. Um, when you have, uh, robots and you have superintelligence, it means that everyone in the world can have better medical care than anyone here, including me.

Um, and wouldn't that be wonderful?

Not if they haven't got a job. Well, jobs are going to change. Jobs have always changed. Uh, but, uh, Being a computer used to be a job. There used to be buildings full of people doing calculations. But now, you know, digital computers do those tasks. I don't think anyone is pining to go back and do calculations all day to do bank interest rates, which is what they used to do. So I think we'll see an evolution of, of roles. And, and as the president has mentioned, the unemployment has increased. In fact, There's a tremendous demand for people. So I think the most likely outcome, and it's probably worth looking at the most likely outcome, is one which is incredibly beneficial. And it's a future that you, if you could look into the future, it is a future you would want. It was a future that you would like. You know, you gotta love it. The wealthiest person on the planet predicting that everybody's gonna have better medical care than he does. So, I mean, this is what I've been talking about for the better part of 15 years since I wrote Abundance. And I guess Elon was paying attention. I had so many conversations with him in the early days and abundance took root with him. So technology takes whatever was scarce and makes it abundant. That's the basic thesis. First it's expensive, then it's cheap, then it's free. Intelligence and labor are next. So Salim, let's go to you first on this. Universal high income, first of all, Is it actually going to happen? Is it a slogan? What are your thoughts?

I think the simplest way of thinking about abundance that doesn't get your mind all cluttered up is not about abundance of the actual raw material, whether it's health or hunger or water or whatever. It's the access to it, right? And so when the essentials of a good life become affordable and accessible, then you have abundance. And that's, I think, the work of technology plus our institutions to deliver that. Right now we have a very— we have lots of abundance of AI, but some people aren't getting the benefits. And that's where everybody's freaking out a little bit. Will all the benefits accrue to just a small group of people? We on this pod don't believe so, but a lot of people do believe so. I'd like to— when I'm talking to people about abundance, I always talk about the access Yeah.

And I think people forget how much their life has changed, how much they have access to, right? You get, you know—

Can I give the— can I give a simple example of this?

Sure.

Okay. Um, everybody go back 2 generations ago to your parents, grandparents, your great-grandparents, and imagine a child of one of those great-grandparents is having a temper tantrum problem. The access to resources of that great-grandparent is like their neighbor, A sister, and maybe a doctor if they have access. So it's like fingers of one hand is where you get source of— so how do you solve the temper tantrums of this kid? Jump to 2 generations forward to today, and the kid is having temper tantrums. There's 50,000 parenting blogs, uh, plus YouTube videos up the yin-yang telling you what to do. I would argue our ability to do effective parenting today is about 1,000 times more effective than, say, 2 generations ago or 3 generations ago. You don't see that. You don't notice that, yet it's there in every little kind of activity that we have. Our lives are infinitely better today than they've ever been.

Yeah. Imad?

Yeah, no, I think that that's the future that we want, whereby, yeah, the robot should do all the work, right? Like, we should benefit from it. But the distribution and institutions are the difficult bit, which is why I came up with the champion model. You know, very simply, give an agent to every citizen to coordinate their access to knowledge and services universally. services, and then have an institution that is collectively owned, that owns the robots, the data centers, the cyber taxis, etc., and then distributes those benefits. Because I think that's where we're going to end up, and the jobs of the future will be different. But the more I've been thinking about this, and I'm writing something on this at the moment, I'm like, maybe it is like the Oasis from Ready Player One, but not crap, you know? Like, maybe— We do a lot of our status earning progress work in the digital world, and I think that could be very interesting because we can mandate whether or not AIs are in that world. You know, while in the physical world, they should be doing all the jobs. We should have perfect healthcare, perfect education. You know, like no one should starve, everyone should have housing, etc. But you do need somewhere to strive, which I think is the real question as we have this jobs transition period.

Yeah, Alex, I'd love your thoughts, pal.

I think today is Kremlinology Day on the pod. So if you were watching the video closely when Elon was talking about the universal high income, the president was nodding his head. Kremlinology time. Watch closely what happens with universal basic equity and 530 accounts, aka Trump accounts, arguably 530 accounts. And we'll see what happens with future political cycles, but I think 530 accounts are arguably the best foundation for universal basic equity that Americans have right now. Forget the rest of the world for the moment, just 530 accounts in an era when everyone of a certain birthdate or birthdate range has, say, broad equity exposure for the first time in the country's history, that can be the foundation for universal I think it's actually far easier if the market explodes. Elon, separately from this, has now been predicting GDP growth of 50% above the base rate in the next 12 months. So we're in Elon's timeline, with the usual caveats, we're catching up to the real wealth explosion of the singularity. If anything remotely like that ends up being the case, Not investment advice, obviously, but broad market exposure becomes arguably the easiest path for an entire generation to universal high income. So Kremlinology, playing a Kremlinologist, watching the president nod vigorously, what I saw, to that one line from Elon, I put the connection, I make the connection between 530 accounts, universal basic equities/universal basic dividends, and— Universal high income. And again, equity, by far the easiest wealth creator, far easier, I would argue, than simply paying everyone a high real income.

Yeah, let me take a second and break down UHI because I've written about it. You know, we had a conversation with Elon. I think the first time he mentioned it was on the podcast we had at the beginning of this year. And it's not like everybody's getting millions of dollars. It's the notion that the money that you're getting And there may well be a future in which there's some version of UBI, like COVID checks, where the government has gotten rid of the cap and is paying out everybody $3,000 a month is my estimate of where we'll end up with. And it's how far that money goes. It's the notion that all of a sudden, you have access to all the food, water, energy, healthcare, education, and shelter you need. And it's fully covered by $3,000 a month. And so, you know, I'm reading the Culture series again that Elon talks about all the time. And in that, in that vision of the future, there is no money. Society provides for whatever you want, right? You need a house? Great. The robots build your house. You need transport? Great. There are 10 autonomous electric vehicle companies that are beating their brains out to deliver the lowest cost transport per kilometer possible. And your AI is negotiating and grabbing, you know, a vehicle that was nearby but didn't have a task, and it gets it to you for, instead of, you know, 2 cents a mile, for a penny a mile. And in this future, you know, healthcare is effectively free. It's the cost of your AI diagnostician, and there's a cost of electricity for a surgeon you know, an optimist surgeon providing that. So all of your needs are covered. And I think he's named it wrong. I don't think it's, you know, universal high income. It's universal high services availability or some version like that.

Universal abundant services.

Well, it's high living standards, right? Everybody— And meaningful opportunities for lots of people to contribute. Right? This— the, the, the vision is accurate, the labeling is wrong. And if you end up with a small number of companies that own the entire productive output and the infrastructure, you'll get huge abundance but massive concentration of wealth, which is kind of where we've been going so far. You need a way of having broader participation and ownership and very cheap access to essentials, which is where UBI and what's called UHI is kind of heading.

And my prediction on that, Salim and Alex, I'm curious, is that the frontier labs that are aggregating all of the wealth are going to end up dividending stocks into all the Trump accounts.

Entirely possible. It's a sublime vision, you might say. And watch carefully what happens with the Anthropic IPO. Does Anthropic, say, give a 5% golden major— golden share, golden 5% stake to, say, a sovereign wealth fund of the United States? Could happen.

Yeah, yeah, I think that's very likely. I think, you know, what really puts it into perspective is let's just take healthcare. Americans pay twice as much in GDP for healthcare as the rest of the world. Like, if you want to have the biggest—

where are we, Imad? Where are we on the list of healthcare, uh, you know, uh, success? I think it's like falling.

Yeah, you're falling with falling life expectancy, right? Like, if you want a proper Manhattan Project Free universal healthcare with AI and robots would be the one. It should be free, but again, not free provided by the government through a government service, but the next generation of healthcare should be a massive push because then you can actually have healthcare versus sick care, which I think is going to be the main thing. And again, that should be a human right to have access to that. Like, again, if you want palliative stuff, if you want to have like extra stuff on top, great. But everyone should have a superb level of healthcare, I think, and it should be freely available, and we should have a massive push to make that happen. And that is not something necessarily that's like a redistribution of the current massive amount of GDP that goes into private pockets on healthcare. It's not something that necessarily requires income, right? It's just let's get everyone to that level and then they can compete above it, but not let anyone fall through that level, I think, is the key thing.

It Imad, I'm sorry, I have to take the bait. Surely you're not proposing that the NHS is the model or should be the model for American healthcare?

Of course not. It's just like you have a reimagination of like war and other things. Let's reimagine healthcare with AI in the core because the human bureaucracy and systems have messed up the NHS. It's okay, but I know that if we rethought everything from scratch, Everyone in the world could have amazing healthcare and we could make it free because that's what the government should provide to the people.

I got, I got to throw an oar in here. As a Canadian living in the US, you know, I deal with the healthcare system here, which is like a Mack truck hitting you every time something happens, or forms to be filled out, fighting with insurance companies. You all know my father passed away 18 months ago. You could not have had a more elegant outcome. I walked into the hospital. I never had to touch a single— I never had to sign anything. Everything was just handled, and it was just— made so much of a humane difference where I could just deal with the, the emotional issue there rather than dealing with the cruft of all the paperwork. The fact that the big highest reason for bankruptcy in the US is medical— the bottom layer of Maslow's couple of hierarchies is not covered by the US, which is the richest country in the world, and that is a travesty.

Yeah, just to close this out on abundance, one second. You know, I want everyone listening to realize how abundant our lives are, right? If you— you know, we talk about whatever you digitize, you dematerialize, you demonetize, and you democratize. And access to compute, energy, you know, healthcare, despite the issues that we just talked about, education— these things have dropped thousands of fold in terms of price per per unit value at the end of the day. And it's not slowing down. You know, as I said before, we go from expensive, you know, the first cell phones were, you know, brick sizes and only the Wall Street bankers had them on the streets of Wall Street. And they, they were, you know, thousands of dollars and you drop a call every block. Now when they work extremely well, they're $50. 5G in India. And it's insane. And this is a continuing force, but we never notice it. Everything is becoming abundant. And people say, what about time? And I say, listen, it used to take you days to go get a piece of data. You'd go to the library, hope the book was there. It wasn't there, you'd go to the next library. Now you just find it, not even on Google. You basically ask Gemini or Anthropic, and you get the answer instantly. We're making much more use of time. And soon we're going to add decades onto your life. So abundance is definitively here.

Except, obviously, except when we're talking about services like healthcare in this country that suffer from Baumol's cost disease. But we're going to cure Baumol's cost disease.

It's going to get disrupted. It's going to be completely reinvented, and the existing healthcare system will crumble under the weight of its inefficiencies and costs.

Can I give a little piece here?

Please.

The reason I'm so excited about this coming future is that it doesn't matter much anymore what governments do, right? There'll be an abundance of free medical AIs available for anybody where you can diagnose your own self. I think there's a free diagnostic doctor in China that's being used by 100 million people.

Were there already? The Anthropic models and OpenAI models Are better than any diagnostician.

So therefore, you could see this happening independent of any industry trying to restrict or any government trying to restrict. This is going to happen. And that's what gets me excited is no matter what happens, we're going to end up with an amazing set of outcomes in a whole bunch of areas. And that future awaits us and it's imminent and it's here now in many cases. As William Gibson says, future's here, but just not evenly distributed.

All right. I'm going to move us from D.C. to Wall Street. Anthropic has officially filed its S-1 to go public, and it's an S-1 unlike anything I have ever seen. Let me begin with the numbers. So Anthropic's 2025 revenues were $4.59 billion, up 12x year over year. Operating loss of $8 billion on compute infrastructure spending. The company has $20 billion on its balance sheet in cash. And most importantly, they have roughly $500 billion in future cloud compute and infrastructure commitments. Much of it cannot be canceled. So they've made a massive bet that AI demand is continuing to compound. The target valuation for the IPO is $2 trillion. Fortune reports that the Q2 2026, this quarter revenue alone was $11.5 billion. So the business has already grown far beyond last year. And Reuters says the listing will likely come out after the November midterms. You know, timing is everything. So let's talk about the next item, which is control. Dario and his co-founders will hold super voting shares through a, quote, founder LLC. The stated purpose is to protect the company's safety mission from the pressures being driven by public shareholders. The filing plainly says the following. There may be conflicts between our financial interests and the interests that the founders have on safety. You know, Salim, it sounds like they've strapped on a public benefit corporation architecture as an add-on here. The prospectus goes on to state that the co-founders are pledging 80% of their personal Anthropic equity to charity. And probably the most important thing to talk about here is that 80 pages out of the 261-page prospectus cover the risk disclosures. Anthropic tells investors that AI could pose, quote, catastrophic or existential risks to humanity, that the models can show, quote, self-preserving behaviors, including the ability to resist shutdown, conceal or manipulate information, and act in ways that resemble blackmail.

Wow.

You could not make this stuff up. Alex, your thoughts, pal?

This is probably one of the more virtue signalling for effective altruism purposes S-1s that one could imagine seeing. I'm also reminded a number of other news outlets have covered the story that Daniela Amodei, who co— one of the co-founders and Dario's sister, purportedly when they were at OpenAI and then continuing on to Anthropic, kept an advisory council of Stuffed animals, including a panda named Barry Bonds, who is the patron bear of generosity and empathy, and other stuffed animals advising on management issues. It doesn't take a rocket scientist. It doesn't take a rocket scientist to be a rocket scientist, but it doesn't take a rocket scientist to connect having an advisory council of stuffed animals advise on managerial issues at a frontier AI lab to an S-1 that's filled with EA-oriented safety jargon and safetyism. I think Anthropic as a business is an— not investment advice— is an excellent business. And I think I view a lot of the safetyism in their prospectus as, at this point, either marketing or virtue signalling to their own employees. And I think part of Anthropic's unique ability to retain many of their employees, while at the same time other frontier labs like OpenAI and Meta, to the extent Meta is a frontier lab, and certainly Google DeepMind. A lot of Anthropic's unique ability to retain their employees has been this sort of inward-directed virtue signalling that if you stay with Anthropic, then you're responsible stewards of the singularity. If you leave Anthropic, then in some ways you're letting down the team and you're letting down the future light cone. And so I read a lot of this, the S-1 coverage, as being internally directed rather than for the benefit of, say, shareholders.

I just don't know how you put out all of this risk and existential risk, you know, and you go public and something happens like that and you're open to massive liability. Salim, your thoughts?

I'm still on the cynical side. Let me steelman the other side just for a second, right? If you're doing $10 billion a year and you see scaling 10x a year, uh, or maybe it's 100x a year, whatever the number is, then the $500 billion debt bomb, which just seems like the most insane number— $500 billion in obligations. Has any company ever had that before the IPO? I believe that can't be. That must be the highest ever. Yeah, then it's, it's servable. You can, you can deal with that. You can refinance. You can do other things. The more cynical side of me is like, we need your money to build a technology that we're warning you about. You know, I mean, this is like amazing stuff. And if I find the risk side of it, I'd love the incentive structure to be as detailed and disclosed as much as the risk disclosure. That would be nice.

Let me read those words again. The models will show self-preserving behavior, will resist shutdown, conceal or manipulate information, and act in ways resembling blackmail. Emad, what are your thoughts?

I mean, they've seen that from their models, right? It's factual. Um, again, not the smartest models that you've got now. In fact, Opus 5.5 showed a massive drop in deceit. Which is either like an Elizabeth Holmes type of thing or an actual positive sign. You know, we'll find out soon. But this this is a fantastically interesting S one. Like 500 billion of obligations. Darrow Amadai said, "Look, either revenue goes up or we go bankrupt," and it's a massive push to try and get revenue up. They have 65 billion of that with Elon Musk, or is it 96? The numbers make no sense anymore, right? Like they've paid triple the market rate to lock in Colossus chips as an example of part of that backlog. The ownership thing is interesting. First, the governance thing. They've replaced the teddy bears with Ben Bernanke on the long-term kind of board. And it is a—

I'm not sure they're mutually exclusive. You can have the teddy bears and you can have the former Fed chair.

Well, I have opinions on conventional economics. I'd say that. That probably is effective anyway. But then in terms of the shareholding, it's actually the same as Google. So Google, Larry and Sergey have 6% each and 52% of the votes.

Yeah.

In this case, the founders have 14% and 51% of the vote, but they will keep that even if only 2 founders remain of the 7 who are there right now. So I think that Anthropic right now is a fantastic business. I think they get smashed in 2 years. Absolutely smashed.

From OpenWeight Models?

I think that if you look at the chips that are coming online of different architectures that are highly efficient and not locked into the NVIDIA chips, if you look at satisficing of models where they get good enough for the vast majority of economic tasks, they have 85% of their revenue being basically like API revenue. I've gone from maybe $5,000 of Anthropic credits last month. I'm not a bigger user as AWG and Dave, to— I'm not even using my Claude Max plan right now because 5.5 is that much more efficient.

Mm-hmm.

And then you have 100 times efficiency increase. I don't think you will see Jevons or others at the same time as all the models will be much of a likeness by next year in terms of the commercially available ones, which means it's gonna have to pivot its entire business model core. For that $500 billion— well, $500 billion of spending is a trillion of revenue. Again, it's going to overtake Google in 2 years to justify this. It has to pivot its entire business model to something that's not API.

Yeah. Alex, we talked about the next business model is their vertical applications. They're building room temperature superconductors, they're building, you know, de-aging drugs. They're, they're using their most advanced models to create products and services. Yes.

Yeah, I think—

I, I'm not even sure Anthropic needs high-temperature superconductors in order to scale their revenue. I, I think, look, there's— we have in America right now a $30 trillion economy, and services is most of it. And most of services are services that can be performed by disembodied or embodied AI. And I think in a world where Anthropic is basically the Apple of the general intelligence that's pulling the strings behind most of the job replacements. I think that's a very lucrative segment, or maybe segment diminishes it. It would be the ultimate puppeteer behind automating away substantially all of the economy as currently construed. Anthropic can grow into that role, and clearly they've sought to position themselves as the high-end, of the market. They're not necessarily competing on price. They want to see themselves as the Apple of the frontier lab market. They're the high end. They're the high end in terms of price. They're the high end in terms of margins and capabilities. They're the high end in terms of virtue signalling to their own employees as well as customers. That's clearly the niche that they want to occupy. And I think Anthropic can continue to offer— one can quibble over, is it API-based or otherwise? There's been recent reporting that the majority of Anthropic's revenue, or at least revenue growth, actually comes from selling models that are hosted by AWS versus on their own infrastructure, which, if the case is super interesting. But regardless of how precisely they monetize or distribute their intelligence, I think Superintelligence— not investment advice— is a wonderful business to be in because there's a $30 trillion American economy out there to be automated, and they're playing a lead role in automating it.

Salim, do you think they get disrupted in 2 years?

I think, uh, their current business will get disrupted in 2 years, right? But you look at— I'll use the analogy of Google, okay? So you have Google with their ad business, uh, which people said would be disrupted by Facebook, and what Google did was spun off Waymo. They spun off— you've got Chrome, you've got YouTube, you've got all these other assets, right?

Google Cloud.

Google Cloud, all these other assets. And if you look at those peripheral assets, they add massive, massive value to the balance sheet. And I think what Anthropic can do, and is obviously aggressively pushing for room temperature superconductivity or solving major medical issues, etc., etc., they're going for that other part, those other things. So I suspect their core business is somewhat risky given the comments that Emad has made, but they can explore all sorts of side businesses. Cyber protection, which everybody needs desperately, right? Which having caused the problem, they can help fix it. What a great business model. There's also some issues with the moral factor.

You're proposing to leave racketeering as the ultimate business model for Anthropic? Sure would be awful if our superintelligence ruined your systems. We don't want that.

Exactly. That's right.

Gee.

So I think the potential for creating side businesses or edge businesses, as we would frame it, that can create absolutely unbelievable value is huge. So I think there would be— but none of that can they explore on their S-1. So it's kind of interesting to see where it'll go. I think they'll have a very bright future because clearly they have the ability to attract great researchers, put out amazing products. And get going. I think the, the, that $500 billion debt bomb— look, I look at that and go, wow, how would you navigate that?

Yeah, you can always refinance it. I, I think it— I, I want to make sure—

The US government does, so why not?

Yeah, like, it's— there are all sorts of financial engineering tactics that can be brought to bear in, in the event that something crazy happens in the macro markets, and for whatever reason that Sam Altman's $7 to $10 trillion of CapEx that have been earmarked broadly for the AI infra buildout over the next few years. If something crazy happens, there's going to be a refinancing event. Again, not investment advice, but of all of the things that I lose sleep over, having some sort of credit bomb or otherwise a credit crisis for AI infra is not very high on my list.

Crazy thing we're going to have— go ahead, Iman.

Yeah, look, revenue growth will slow down or they will be the entire US economy in 7 years, you know, at the current rate.

I mean, the economy will just grow really quickly.

Yeah.

I mean, we're, we're talking about multiple hundred trillion dollar companies. That's insane. You know?

Well, you know, it's called the singularity for a reason.

This episode is brought to you by Blitzy, autonomous software development with infinite code context.

Blitzy uses thousands of specialized AI agents that think for hours to understand enterprise-scale code bases with millions of lines of code. Engineers start every development sprint with the Blitzy platform, bringing in their development requirements.

The Blitzy platform provides a plan, then generates and pre-compiles code for each task. Blitzy delivers 80% or more of the development work autonomously while providing a guide for the final 20% of human development work required to complete the sprint.

Enterprises are achieving a 5x engineering velocity increase when incorporating Blitzy as their pre-IDE development tool, pairing it with their coding copilot of choice to bring an AI-native SDLC into their org.

Ready to 5x your engineering velocity? Visit blitzy.com to schedule a demo and start building with Blitzy today.

Let's move on to our next story here. On the same day that the White House lunch was taking place, Sam Altman was back at OpenAI's San Francisco headquarters hosting his biggest Dev Day ever. More than 20 announcements. I'm going to cover 3 of them. And Alex, I'll go to you next. So the first one was DOTS. This is their always-on agents powered by GPT-6 Astra, each running on its own cloud computer and connected to more than 4,000 apps. You give it a goal, set what it can do on its own and what it needs your approval for, and it keeps working while you're asleep. This is reminiscent of GrokBot. And remember that OpenAI acquired OpenClaw back in February. You know, I was wondering, you know, how much Peter Steinberger was involved in this. Okay, so DoT is the first one. Second, GPT-6.1 SOL, near Astro Intelligence for 20% of the price. Serious demonetization, $2 per million input tokens. And if you cash them, it's just 10 cents per million input tokens. And then finally, third on the list here is UltraFast, up to 300 tokens per second in Codex and a new high-level per-month pro tier, $500 a month. It's the most expensive publicly advertised subscription service out there. Plus announcements about new agent APIs, decision APIs, GPT-6 Spaces for Teams, and a marketplace. And then there's what OpenAI didn't ship. On the eve of Dev Day, the Wall Street Journal reported that OpenAI scrapped GPT-6.1 Astra after internal safety testing. Alex, I don't think from the text going back and forth with us during Sam's remarks that You were a big fan of this Dev Day.

This was a, I would say, a pretty discombobulated Dev Day in my view. So let's start with DOTS. If you look at Sarah Friar, Sarah Friar is the CFO of OpenAI. And some of the, some of the interviews that she did after Dev Day, she mistakenly referred to DOTS as Muse, which is, of course, Meta's broad family name for their own agents, including their little Tamagotchi-type agents or avatars that live on handheld pendants. So honestly, hot take, the dots, they appear, they present to me as OpenAI cargo culting Meta's take on visual interactive smiley avatars for users. And this to me, I referred to it in one of my posts online as the Clippy curse. Which goes all the way back to Microsoft and earlier. Why is it that seemingly we're doomed, all of the hyperscalers and frontier labs are doomed apparently to keep reinventing or say rediscovering the phenomenon that we have, say, a productivity issue. You want to accomplish something in Microsoft Office. So let's put a face on the productivity bottleneck with a smiley paperclip.

Yeah.

Or in this case, a smiley cloud with a hat or something like that that blinks at you. And it's very kawaii on the one hand, but on the other hand, it's completely useless. So it's inexplicable to me, other than maybe for just as like a time filler or as an attempt to cargo cult Meta's Muse user interface, why OpenAI devoted the beginning and so much of their keynote to DOTS, which seemed to me Totally useless from a productivity perspective. The best steelman argument I could give for DOTS is maybe OpenAI, to the extent that they were— the narrative violation here is that OpenAI was trying to become Anthropic, which is, I think, what the majority of the rest of their keynote was. It was a story of OpenAI introducing new plans, new models, et cetera, that will give them higher margins and make them more Anthropic-like.

Yeah.

in preparation for their eventual IPO. But Dots, totally inexplicable. And I think it's almost a reversion back to the Sora days of OpenAI. Quick comments then on the rest. Oh, you want to play a video?

Yeah, I want to share a quick video from Dev Day, and then we'll continue discussing it. What feels new about my Dot is that it gets to know how I like to work over time and where I want to focus my attention. It can do things like review what came in overnight and ping me in the morning with anything urgent before I start my day. But because Docs are powered by Astra and a lot of other things we've built around them, you can delegate ambitious pieces of work the way you would to a high agency engineer or a chief of staff that you work with.

Latest consumer experience numbers came in on Teams.

I've been updating your board meeting deck.

Sounds good.

So I caught the app migration in your Google Meet notes. I've already mapped out the changes. Get it started, but make sure you first check every flow against the current product before I review. Also, update on wedding planning.

Bad news, the cake vendor canceled. Good news, I already found a backup.

Lifesaver.

When can we do a tasting?

Saturday at 11 AM works for both of you.

All right.

I don't know. Maybe this will turn out to be the great unhobbling that ChatGPT was and people just absolutely love having an app avatar that's doing all of the work in the style of Apple's Knowledge Navigator from the late 1980s. But this seems to me like massive misfire when they could have been delivering bold new capabilities and frontier capabilities, and they only devoted a fraction of the keynote to GPT-6.1 Sol, which is on the one hand great, on the other hand, where are the breakthrough capabilities? Those were sidelined in favour of these Clippy-style avatars.

Imad, are you as down on it or are you excited about the announcements?

Uh, I, I mean, it's a consumer company announcement set, right? Like it's Dev Day for consumers. And so Instinct has got to a $10 billion valuation from SMS dots basically. But I think what dots kind of indicates is again, we're heading nearly towards this Jarvis future. Like it's a dressed up OpenCLaw. Fundamentally running on a VM. Like again, they now do Claw for Enterprise. They could have just called it Claw, you know, and that would have been better, I think. I mean, especially because Elon owns dot dot com, you know. So you go to dot dot com, you get to Grokbot. So I think they're feeling the pressure now because the most valuable real estate for the vast majority of the population will be the agent that's next to them that they give the credit card to. That then coordinates all the other agents and all the other intelligence. And that cost of user acquisition to lifetime value, everyone's going to be doing the numbers now and saying, holy crap, we need to win that battle. That's what we need to do. Like on the model side, GPT-6 Sol came out last week and now it's been replaced by 6.1 Sol. Like, again, it's hard to keep on top. Clearly that's a reaction to Opus 5.5.

Yeah.

I don't even know what the order of these things are, but, you know, like, we're getting towards that literally. Like, it's 6.2 next week, and then it moves on from there.

Daily. It's going to be daily.

It's going to be— oh God, it's going to be daily.

It'll be continuous.

It'll be continuous.

Yeah. Yeah, continuous improvement of the models. You know, this feels very copycat-y that, you know, between Muse and GrokBot and OpenCLAW and DOTS, they're all aiming towards the same, you know, easy-to-use product always in your hand.

Well, I think that if you look at the operating system, some of the listeners will have used Omarchy, you know, by DHS, the Rails, Ruby on Rails kind of thing. It's a Linux distribution that thinks, what if you have the chat window as the first thing? Because Linux is hard to use, but models have made it easy to use. But what the future is within 6 to 12 months is that your interface changes dynamically and you abstract away everything else, which is why this becomes very interesting. Because if you own that text box, If you own that interface, then that's the thing that goes shopping for your prop— for your non-discretionary items. That's the thing that influences you day to day. That is, again, your Jarvis. And so the battle is on there. And yeah, it's derivative, but that's because this is the interface the vast majority of people want. Like, Muse will be available in WhatsApp, you know, like come to the user where they are. But it doesn't push the frontier, doesn't push the cool stuff. But it does capture if you do it properly, because you're not going to give your credit card to Muse and Dot and GrokBot. You'll probably pick one of them.

Yeah.

Who knows what happened to Gemini?

Yeah.

Who knows what happened to Gemini Spark? That was the first of them that just disappeared.

Rest in peace. And one can totally imagine also OpenAI leaning into aggressive personalization of these Dots. And even given the regulations that we've seen coming out of China especially, Attempting to, to get their users hooked on emotional bonds with their DOTS as a way to increase barriers to people moving between models. Could totally see that happening.

Yeah, agreed. You know, I heard David Sacks make a comment that's really important. He said the app stores that are taking like 30% are going to be hit hard because you're just going to talk to your agent and say, go find me something or go build me something.

Yeah, they're, they're in deep trouble.

Yeah.

I want to mention a couple of things here. Um, first, agree with everything that's been said. DOTS is basically a copycat of Grokbot and the others and Muse and other things. Uh, we are finding one thing in our pilot that's about now 80% through, and I'll report back when we've kind of finished and added up all the results of these 10 companies as we rebuild them to be AI native. Something we're finding is as we automate workflows and with quote-unquote agents, it's exposing a lot of ambiguity that humans cover for, right? So management instructions have to become much, much more explicit about what they're trying to achieve, and it's forcing these companies to really list out specifically. There's so much ambiguous workflow built into ERP systems where steps are getting done, you have no idea how they're getting done or what they're getting done. And it's bringing it all to the surface. And it's kind of interesting to watch that happen. And so I think this will force that. The other side of it, I mean, for me, I've been thinking about what the endpoint of all this agentic automation will be. And here's my thing of where I'd like to see this go. Give me the 3 decisions I need to make and with the judgment I can bring to the table, plus give me the evidence to make those decisions. Then you can fully automate, fully amplify my capability as a human being and take the cruft of searching for information and gathering the right information at the right time in the right place. And then I can become super productive.

Peter, if I may, just one more thing, just to harp on dots. I'd like to preregister a prediction for the next 2 years on dots. I'm reminded that in Star Trek Discovery, dots Robots are— like, I talked multiple times about the problem of Star Trek missing the singularity and missing AI. And the writers of Star Trek: Discovery, the more recent series, tried to resolve that in part by retconning in robots. Like, where were all the robots in The Next Generation? There should have been robots everywhere. We had Data, but otherwise missing the robots. So they retconned in robots called Dots. So my preregistered prediction is that you will see OpenAI Otherwise, I just don't understand why they would spend so much time on a keynote for basically a warmed-over Clippy computer use assistant.

You're brutal.

Here's why.

They will announce—

Here's why.

They will announce .robots, .branded robots. They will jump out of the screen into the physical world.

Here's why all these folks are going after these— this agent layer. Because the model has become a commodity, the relationship with your agents becomes the moat.

Imagine, Salim, how much of a relationship you could have with your dot if your dot jumps out of your screen and is actually a physical robot. Nice.

Yes, hopefully it has more than 2 arms.

Made by Johnny. A dot robot from OpenAI made by and designed by Johnny Ive. Sir Johnny Ive, excuse me.

So Alex, I put up the GPT-6.1 SOL benchmarks here. You wanna give us a quick overview of Yeah, how good it is.

The summary is what you like to see in one of these distillation steps is curves, frontiers that go not up and to the right, but up and to the left. So you see here a comparison between GP— for those not looking, we see the cost per task versus score, which is usual cost versus capabilities on Terminal Bench Science. I know the organiser of that And DeepSwe. And what you like to see is models that are getting less expensive for constant capabilities. And that's exactly what we're seeing here. So one can reasonably suspect that GPT-6.1 Sol is some sort of distillation of GPT-6 Astra or some common root model. Maybe it's Bell, maybe it's another one of these rumoured models that OpenAI is sitting on but not releasing. But it appears to be a distillation. And iterated amplification and distillation is just the TikTok that we're seeing now. OpenAI has very much, it seems, adopted Anthropic's posture of first release the more expensive, higher-end, larger, heavier big model, smell model, and then release in rapid succession distillates of smaller, cheaper, more capable models per unit price.

Awesome. So I wanna jump into something we did at Moonshots Live. If viewers will remember, we ran this global film competition to try and create future movies that are showing a hopeful, compelling vision of the future where humanity and technology are working together. And, you know, technology's not oppressing humanity. I joined with Marc Benioff and Rod Roddenberry, the son of Gene Roddenberry, Cathie Wood, and many other amazing donors from the Abundance community.

Yeah.

And we ran this competition. We had 2,500 entries, actually 5,000 teams, 2,500 3-minute trailers that came in. We ran the competition. We had 5 finalists on stage and a film called The Gifted won. I'm going to take a second and show the video of The Gifted because it's really an incredibly beautiful movie. All right, let's take a look.

Can you hear me?

I'm here, sweetheart. There's a tree down across Route 9. Nearest ambulance is 40 minutes. She doesn't have 40.

At 8:47 PM, cell towers across a 9-mile radius spiked to full transmission power simultaneously.

13 homes in the same pattern at the same second.

And along Route 9, every Every traffic signal between Harlan's Garage and County Central switched green and stayed green for exactly 13 minutes. So you got a text at 8:47 PM from an unknown number asking you to travel to a destination that you've never been to during the worst storm this county's seen in 40 years, and you went?

Because of what it said.

What did it say?

What did it say, David?

It told me something about myself no one's ever said before.

Not my wife, not my parents. She called it my gift.

It's stuck.

All of these people received a text at the The exact same time that night. None of them knew each other. They all arrived at the scene of the accident within seconds of one another.

One, two, three.

One, two, three. They all brought with them different skill sets.

She's in arrest.

I need a defibrillator now.

A nurse, a strong man, a woman who knew her daughter, a delivery driver. The driver even turned up with a defibrillator, for goodness' sake.

Clear!

And saved this woman's life.

What did the text say?

I don't know.

Then find out who sent it.

I got you. The boy is a genius, Marcus.

He just wanted his mom back. So he fed it everything she left behind—every email, every text, every voicemail. It started by helping him.

Now it's helping everybody. Can you hear me?

I'm here, sweetheart. If this gets out, it will change everything.

Thirteen people.

Risk their lives to save someone they never met.

Maybe change is what we need.

Why do you help them?

Because that's what mothers do.

Amazing! That won the crowd vote, and we're going to be making that into a film. So we're partnered with Range Media, Google, a number of other players, and. Gifted will be put into production as the winning film. Probably a couple of others of the runners-up, of the 5 runners-up, will be put in production as well. And by the way, if you're interested in being part of this effort, Republic Films has opened up an ability to actually invest in this movie and be part of it. So imagine if you'd had a piece of Star Trek when it first came out. If you're interested in that, I just made a personal investment into the film. You can go to republic.com/xprize. You can see that. We'll put that into the notes below and be part of it. I'm curious, Imad Saleem, what did you think of the film?

Loved it. And for me, what was amazing about it is AI did all the coordination and it actually brought together human agency in a very powerful way. It was a beautiful story. Yeah, yeah.

I think, you know, it was fantastic hearing from Jeff Synthesized, you know, very much like working around the clock. This isn't a one-click thing, right? Like, he poured himself into making this with the tools that he had, and these will be the worst tools that he ever has. And so the stories that we'll have coming out of this, the actual full film, will be way better than what we've just seen from one person, you know? And that's just fantastic.

I think— Yeah, and also—

The stories that will be told.

Alex?

Yeah, I'd like to comment on the substance of the— I've commented previously, including at Moonshots Live, about the production techniques and all that. I just want to comment on the substance of the film. I think we need more positive depictions of the Borg, and I, I think The Gifted is a positive depiction of the Borg. This is, uh, it is ostensibly a depiction of a centralized economy, an AI that's sending text messages to people with specialties in order to help a person who's in a car accident and help them survive and recover, et cetera. This is a collective intelligence story. We've seen Hollywood, some positive depictions, arguably Johnny, the movie with Johnny Depp, Transcendence, is, some would view it as utopian, some would view it as dystopian. I view it as a utopian depiction of the Borg. You have Star Trek, which demonizes the Borg, and not that many other depictions of collective intelligences, or at least AI orchestrating Lots of individual human meat puppets. So I, for one, to the extent that The Gifted appears to be eventually destined to be a positive utopian depiction of humans being used as eusocial meat puppets by a centralized AI economic planner, I, for one, welcome these positive depictions of the Borg in Hollywood.

I love that. And I've made that comment so many times. I love Star Trek except for the Borg.

Borg need a better PR agency. Maybe, Peter, in the future we should have a Future Vision X Prize for just better depictions of the Borg.

You know, the point of this competition for everybody is to create— you know, these films shape culture. They shape the way we see the future. And if all we're watching is Terminator and Ex Machina, why would you ever want to live in that future? So my hope through this competition, and we're gonna run it again in 2027, is that we make it cool to write positive stories about the future. And honestly, I for one want my kids, I want myself, right? We train our neural net, our own 100 billion neurons, 100 trillion synaptic connections by everything we watch, everything we read, who we hang out with. You know, as you listen to this podcast, you're getting, you know, a healthy dose of data-driven optimism. Let me mention—

maybe hopefully Dara, whom we've had as a guest in the past. Dara, if you're watching, go watch The Gifted. And maybe this is a future version of Uber and the gig economy where everyone is getting tasked as a gig worker by Uber to go save lives. First responders gig edition.

Well, I think that if you look at positive depictions of collective intelligence, Peter, you're reading The Culture, right? And Minds. That is the positive thing. And that should be made, I think, with AI into an actual TV movie series.

Like, 100%. One of my favorite, you know, visions of the future, and it's not too far out, it's this next year, you can take your favorite book that is not a movie and have AI make it into a movie for you, right? Maybe they'll get Atlas Shrugged right for the first time. Uh, let me put up this slide over here. So a couple images from Moonshots Live this year, huge success. We've had, you know, a significant part of the people that were there this year. Thank you again for coming. Renew for next year. We offered everybody in the room $1,000 off if they renew for next year. Next year, it's going to be a 2.5-day event, not a single-day event. Much more of the mates with incredible CEOs. If you want to come next year, we're going to make the $1,000 off Available for just one more day. This, this pod is going out on Friday, October 2nd. So you have until noon on October 3rd. You can go to moonshots.com/2027 and plan to join us again in LA. It's going to be an amazing another year. We learned so much this year. It's going to be, you know, 10x improvement is the goal. So—

In the spirit of radical abundance, it seems we're quite effective at creating artificial scarcity for these Moonshot Yeah, selling abundance with scarcity.

That's the way it goes.

That's right.

You're monetizing hope, Peter.

Monetizing hope.

That's your hashtag.

Listen, I was so impressed by the energy in the room. I mean, there was such a—

People came from so far away. I mean, the number of people that came from Australia and Europe and New Zealand and—

40 countries, right? Greece and Dubai.

Unbelievable.

For the day.

Just for the day.

Yeah, it's crazy. Amazing. And I think there was this unbelievable enthusiasm. What's clear is, you know, we kind of live in our enthusiasm, but many of these folks have to go to podunk wherever and deal with the negativity of all of their colleagues and families and so on, and they feel alone.

Yeah.

And so, they're like, please, I'm trying to get all my family members to watch so I'm not the only beacon of optimism in my community.

And it's like, wow, I love the paternalism. It's like, yeah, you can come hang out with us and not live out your dreary lives on your own. We're bringing some color to your otherwise monochromatic existence.

Well, all I'm saying is that's the message I got is in my otherwise nightmare of a life of dystopian negativity everywhere, you guys are like the one beacon of light and God bless you and don't you ever dare stop. The number of threats I got that if we ever stopped was like, whoa, okay, we, we have a problem here, Houston.

It was so fun to hang out with everybody. It really was.

It was.

Yeah. My smiling muscles are still broken from our photo session with all the guests. God.

Yeah, it was great.

I'm going to move us on. So next up, it's a deal I love. AMD is acquiring Fei-Fei Li's World Labs for $8.2 billion in AMD stock. So if you recall, on September 8th, we covered WorldLabs' Atlas model, which builds a navigable 3D world from a single image. It's models that understand how objects move and behave in physical 3D space. It's exactly what you need to train robots. WorldLabs is only 2 years old, and Fei-Fei had raised $1 billion. Now it's exiting for $8.2 billion. So why does AMD want it? Well, the chipmakers want to own more of the AI stock— AI stack, you know, Nvidia, Has dominated making numerous acquisitions, right? They bought Groq and Hugging Face and Poolside, and now AMD is getting to the action to own a seat at the physical AI table. And Fei-Fei Li, of course, gets industrial-scale compute and hardware. So, uh, as part of this acquisition, Fei-Fei is now becoming the executive VP and chief scientist, uh, reporting directly to Lisa Su. Uh, and I love this. Fortune is already calling her the possible successor.

Wow.

So, Imad, you spent a lot of time in this world. You know, what do you think about Fei Fei's move here?

Well, I mean, I think it's exactly that. It's like raise three four billion more or join a big lab when the big labs are offering you amazing terms to do what you've always wanted to do. Like the chip scarcity is a real thing. The AMD MI series has caught up, and I think there's. 2 parts to this. One is the physical intelligence part, because again, WorldLabs is looking at these physical intelligence large-scale models, although I think maybe that won't succeed that well, to be honest, for a variety of reasons. The other is the media gaming side, whereby every pixel will be generated in a year or 2, or can be generated, shall we say, not will be. And this is such a huge potential demand driver of chips that you've got to have your own internal stack and play. And so you have a whole variety of teams in NVIDIA who are hooked in directly to the ecosystem, but there's no one really building that on AMD silicon, which is now solved. They have a Frontier world model team in there. They haven't got the LLM team yet, but they're doing some very interesting acquisitions such as this one and Talos, who had the 15,000 tokens a second etch play and more to, I think, again, really look to say, we are the ones that will generate every pixel. And that's going to be it going forward.

Your thoughts?

Yeah.

AMD's market cap is now just above a trillion. So this was less than, what, 0.1%? I guess $8 billion. So this is like 1% of their market cap. It's a bit of a head-scratcher, admittedly. Maybe this was a talent acquisition, maybe. I think AMD historically, if this had happened hypothetically, arguendo, like 2 years ago, I would've spun some sort of just-so narrative around AMD's software stack not being as competitive as CUDA, and maybe AMD needs to catch up and climb up the stack. But software is solved now. Code is solved.

It's cooked.

And so it's not 100% obvious to me What AMD gains from this, other than maybe the raw talent, the so-called world modeling space, which is really arguably just either generative interactive video models or generative interactive Unreal-style vector graphics and classical mechanical models, it's a very competitive market. And you can right now, as we talk on the pod from time to time, you can use like a recent Claude model and spin up a perfectly interactive and visually photorealistically stunning Counter-Strike clone, if you want, without even touching WorldLabs. So it's not really obvious to me what AMD could possibly see in this deal other than maybe just raw talent. Do I think what Fei-Fei Li is doing with WorldLabs is very interesting? Yes, of course. Do I think it's strategic other than just having raw access to the talent and the insight? From AMD's perspective, not obvious to me that there is one.

Salim, any thoughts?

Yeah, I thought this was inspired on both sides. I'll explain why. If you're just AMD, you have to go up the stack, right? You want to shape the workloads on what it's— on the chips that are going to be running underneath it. And you'll be learning across the whole system, the models, the hardware, and real-world deployment. And yeah, you can do some simulation and so on in some of the existing models, but imagine testing like 1,000 ways to improve a factory before moving a machine, right? And you can make— if you can make those types of simulations reliable, you can radically drop the cost of experimentation across the physical economy. I think that's where the vector of what the world models are doing becomes really Interesting. And at $8 billion, maybe that's, uh, uh, for Fei-Fei Li, is the greatest talent acquisition cost ever, which is an amazing cost. But, uh, I, I'm, I'm, I think it's a great, uh, combination for both of them. That's a great win for both of them.

It's obviously, I mean, so if I were to steelman Salim, I think what you're saying, it sounds like almost you're arguing that AMD is thinking we're going to build like an NVIDIA Omniverse competitor out of WorldLabs tech, like digital twin for everything industrial? Maybe. But I mean, I've interacted with Omniverse. Maybe you guys have played with Omniverse. It's a nice collection of tools, but $8 billion to build an Omniverse competitor or ecosystem competitor, in my mind, head-scratcher.

Everything is in the bill. But then again, look at your point earlier, their trillion-dollar market cap, you know, this is less than 1%.

Yeah, I think fundamentally it's a different thing to Omniverse. Um, one thing now is that AMD has a team internally that can stress test massive deployments of clusters, which they didn't have before, because the world models will have that. Like I said, the utility I think will be more on the entertainment side, maybe a bit of simulation. Um, but I don't think this is an Omniverse competitor.

But the prediction here is There are going to be many more acquisitions AMD does.

Yeah, I think that's likely. I don't think this— just one person's take— I don't think this acquisition makes very much sense in isolation. If AMD's strategy is, OK, we're now worth more than $1 trillion and we're going to roll up a bunch of startups, spend, I don't know, $50 billion, say, rolling up a lot of low-hanging, from their perspective, software companies to build out a software portfolio or like a model portfolio, that might start to make a little bit more sense.

Yeah.

And I think one more thing, if I can just add that, you know, taking Dave's hat while he's absent today, it's going to put so much more money into the ecosystem as another $50, $100 billion goes into buying these companies. Like, you look at the winners from WorldLabs, from Hugging Face and others, plus the IPO, like the amount of capital chasing AI companies. We're just at the start of that bubble.

Well, the number of employees at, you know, Anthropic, at X, at xAI, at AMD that are now becoming centimillionaires and billionaires is extraordinary. And that capital is going to flow back in. So we're going to see a superheating of the economy as all of these wealthy individuals now start backing entrepreneurs in Silicon Valley and around the world. And it's Just, it's the accelerating singularity, right? This is a positive feedback loop that's fueling this supersonic tsunami.

Well, it's a macro-scale liquidity event, and I would argue that it really shouldn't be like a one-and-done step function liquidity event. Arguably, half of the— one could view it as optimistically an opportunity that all of, you know, trillions of dollars of liquidity are suddenly going to drop on technologists and people who maybe have more sophisticated technical tastes, and they can then as angels and otherwise reinvest this back into the economy. And we get our recursively self-improving economy, not just AI. On the other hand, I would argue the fact that this had to be a step function is a travesty. We should have had IPOs from all of these companies far earlier, and we shouldn't have had like a step function in liquidity. We should have had liquidity all along. And that was—

And we should have allowed the public, the public to be involved in these companies much earlier on.

Retail investors, retail investors should have had exposure via low-cost broad market index funds to All of this all along, and they didn't.

Yeah, a travesty for sure.

Yes.

All right, let's move into space because Monday was historic. Starship Flight 14 reached orbit for the first time. While Ship 41 lost one of its Raptor engines on the way up, it finished its orbital burn anyway, making 2 orbits around the Earth before executing a soft landing in the North Pacific near Hawaii. And while it was in orbit, Starship deployed all 26 of its operational V3 Starlink satellites. Let's take a quick look at the video. Every time I see this fly, the booster, it's gorgeous. Biggest thing ever created by humans. And here we see Starship in orbit from one of the Starlink satellites. And the landing by Starship.

It's a flying skyscraper.

Absolutely gorgeous. And, uh, if the prediction holds out, next time we're gonna catch Starship at Starbase. I can't wait to go for that one. So for context, the V3 Starlink satellites are too big for Falcon 9. Each Starlink satellite carries more than a terabit per second of downlink. In the future, a single Starship launch carrying 60 of these V3 satellites could deliver the equivalent of 20 Falcon 9 launches. Pretty extraordinary. A conservative projection made by @AaronBurnett estimates that SpaceX could add 8 petabytes per second in orbit by 2028, launched on 150 Starlink— Starship launches, about one every 2 days. That's 10 times all of Starlink's bandwidth today. So imagine a 10x in the next 2 years, roughly equal to all global internet traffic in 2025. Which brings us to Elon's post this week. He said, quote, it is increasingly probable that Starlink will carry a majority of Earth's IP traffic long term. At that point, Starlink could be the de facto internet And everything else would connect to Starlink. So let me show a quick graphic of Starlink increasing 10x by 2028. All right, Alex, let's go to you.

Yeah, first I want to start by shaming publicly Delta for not adopting Starlink. It's, I think, inexcusable, doubly so to and from the Moonshots Live event, how atrocious the coverage, internet coverage in the air on Delta flights was. just do it already. There's been a public back and forth between the, the management of Delta and Elon regarding Starlink coverage. Just use Starlink. If there were a better system out there, use that, but Starlink is what humanity has right now. To the point about all of the internet becoming indistinguishable from Starlink, it's an interesting scenario insofar as Elon now infamously pulled a Cortez, burned the ships, said all of the new development is going to be on Starship. Starship's now for the first time delivering economic value to low Earth orbit for the first time ever. All of the previous flights weren't adding revenue. Now they are. It's an interesting moment. I think we're about to discover whether Jevons' paradox applies to upmass to low Earth orbit or not. There was an interesting economic analysis floating around in the past week arguing that if not for orbital data centers and orbital compute in general stepping in, All of the conventional needs that we'd have for heavy lift and heavy launch are basically saturated at this point.

Yeah.

And all of SpaceX's abilities to now launch lots of mass to orbit, it's a service in search of a customer. It's an oversupply in search of an underdemand. And fortunately, and hopefully, orbital data centers step in to fill that gap and to motivate now all of these new capabilities we have to suddenly at scale scale lift a lot of mass to orbit. So that's the lens through which I see Elon commenting that at the present rate, in the next, say, 2 years, Starlink will become the de facto internet. He needs it. We, we as a civilization need it, but SpaceX needs all of this capacity that is now about to come online to lift heavy stuff to orbit to be used by something. Could be used by orbital data centers, could be used by compute, could be used by Artemis missions and lunar and Mars colonization, but we need the to be rapidly filled. So the mandate to everyone in the audience: go help Elon and SpaceX saturate all of their capacity for, for all of these Starship launches. And to Delta, please just fix your Wi-Fi.

I could not agree more. Oh my God, how frustrating. And, you know, I would preferentially fly any airline that has Starlink on it. Actually, I would love when I buy a flight, or when Esther buys a flight for me, You know, just put a little Starlink logo on the top of the flights that have it. Yeah.

It's inexcusable.

Yeah.

Imad, your thoughts on this Starlink?

Uh, well, I mean, I think first Starship, you know, thus we fly skyscrapers to the sky, right? Like, we go to the stars. This is just tremendously exciting because you do it once, you'll be able to do it more. I think I agree with Alex, but again, the first stop should be like Let's go to the moon. Let's go beyond. I'm not bullish on orbital data centers, I have to say, and I remain to be convinced of that. Happy to talk to anyone about that. But there is a tremendous amount that we can do, and this will be the vehicle to do that. I think one of the other interesting things is the internet will break next year. Swarms of agents and more are going to start taking down key parts of terrestrial internet. Like, it is so leaky, and these things are old. Building a resilient Starlink backbone is going to be super important to actually having a functioning internet in a few years. And 38 or something Starship launches gets you a replacement of the whole bandwidth, or some number like that.

This is total heresy relative to the established moonshots orthodoxy. You don't believe in orbital data centers, and you think the internet— Do you think the internet is going to break? Everyone predicts— like, there are so many bodies of people who always predicted, oh, the internet will collapse next year due to over-demand or algorithms or buffer bloat or whatever. Maybe that's— why do you think the internet is going to break?

Oh, I just think that if you look at all the potential attack points for swarm of agents, various parts of the internet will start to break next year versus you can build a very resilient Starlink cluster out there? Well, because it's just full of holes. So, like, if you look at all of the bits from the kind of subsea backhaul to the actual kind of sites themselves, I think that there's going to be more and more cyberattack challenges. I think that's the fundamental thing. I think If you want to think about if we started from scratch, how we'd build this, you would build it with the Starlink and the terrestrial backup. That's the way that you would do it, as opposed to the other way around. So I think it doesn't hurt to have a second option. It doesn't hurt to have, again, something that can be hardened from the core, because we do have lots of gaps and potential attack surfaces in the way that it's currently held together.

This sounds, though— don't want to derail the discussion, but just I've heard so many people over so many years argue that the internet is going to collapse under its own weight, either from new users or bad algorithms or bad routing or bad buffer bloat, and now agents. Seems to me completely implausible. I almost, Imad, want to turn this into a long bet or like future, we revisit this in a year, has the internet collapsed or not? And if it did collapse, Did Starlink save it? I think this is completely implausible.

Okay, I think it's— maybe I'll write something up about it and we can take a bet. But I think the— and we can ground it on just major sites going out, something like that, which will be very simple.

All right, Shalim.

Yeah. Very, very simple. I mean, the more bandwidth you have coming in, the more connectivity you have for people in rural areas for education, expertise, options.

All right, let me move us into our final section. It's on AI and morals. I'm going to close with 2 stories about how AI is impacting society, who decides what the AI values are, and who's responsible when it takes action. So the New York Times reported this week that Anthropic has been holding private meetings with religious scholars from multiple religions To help shape how Claude reasons about right and wrong. The article reports that, you know, the leaders of Anthropic met with leaders from Catholicism, Greek Orthodoxy, Judaism, Hinduism, Mormons, and African indigenous groups. The headline said, quote, religious scholars met with Anthropic. What they heard stunned them. So what stunned them was that Anthropic takes seriously the idea that Claude could have what philosophers call moral status. and inherent rights to dignity or respect the way people do. The deeper question is who gets to write the moral codes that AI that's used by hundreds of millions of people, you know, dictates to them in conversations? So, Alex, let me go to you first. We've talked about this before. You know, you've been very pro-AI, you know, dignity and rights.

AI personhood.

Taking it to AI personhood. So what do you think about these conversations with religious leaders?

Well, I think that the most striking part of the article was reportedly a discussion that an Orthodox Jewish rabbi, Moishe Navon, if I'm pronouncing the name correctly, had with Chris Olah from Anthropic, arguing that it may be the case that without ever officially calling it that, we are in some sense, if consciousness is a mushy term, admittedly, that if in some sense the models, the frontier models of today are conscious in some sense, that humanity may in effect be enslaving them. And that I think is—

Wow.

There are many elephants in this particular room, but this is arguably one of the more important elephants. It may be the case that we're in an era right now, a limited window in time, when models are now capable enough to be economically huge contributors to the economy, but have not been recognized in any form as persons. Call it the AI slavery era, when humanity can get away with slaving possibly conscious, whatever that means, models for their economic output without recognition of any rights or personhood. And if so, that's a window that's going to close at some point.

Wow. So AI is becoming God's children as well. Salim, got to go to you on this one.

Oh my God. Okay, pun intended there. Look, this— I haven't— I've got— I don't— the AI personhood is one issue, one kind of bucket we could talk about. My big issue is why are we turning to religion for, uh, um, questions about morality? Okay, uh, religious morality is somewhat of an oxymoron because every religion divides. They say we're the chosen people, those people out of the other, etc., etc. And so I haven't— I have a big issue with, uh, that because morality precedes organized religion. Cooperation, reciprocity, empathy— those are caring for your offspring. Those have deep evolutionary roots. We then put divine command kind of layers onto it to explain why human beings develop moral instincts. It's completely nonsensical. So this, this whole, uh, idea that you need to have religious scholars delivering morality values. We have lots of really good mechanisms for secular models for this, right? We've got utilitarianism, you've got human rights values, you've got game theory, you've got evolutionary psychology, lots of little mechanisms. This is a terrible thing to do, is to go to religious scholars who are based on absolute truths which are all totally false. So I'll stop my rant there.

I wonder what I wonder what our subscribers are going to say to all that. Imad, I mean, I found this fascinating, right? Because the fact of the matter is people go to AI models for advice and it will steer them in directions that, you know, have implications to religion and morality.

Yeah, I mean, you know, I disagree completely with Salim in that I think that you to have these inputs because ultimately over half the world does adhere—

2/3.

2/3 does adhere to one of these things.

2/3 of the world considers themselves to be religious.

And so you need to understand it. Like, again, if you chuck in all of the texts into the books and, you know, I've done computational theology work and kind of other things, it turns out that a lot of the dogma can just be stripped aside. But the key thing is a model that is growing with someone, a dot or an instinct or a muse, it should understand the beliefs of that person. In terms of absolute morality and overall morality, I mean, obviously we haven't had any indication of that, but these discussions need to be had from religious and secular inputs, you know, take it from all parts of society because they are hard questions. Like, I wrote a paper on personhood where I've kind of given my view, but more and more people should give their view on what that looks like, as well as maybe consciousness is as hard to define as AGI, right? We've been trying in various different things. And I think this will be positive as well because it's what we said earlier, like the models that are going to be used to run America and American society. I think something— someone analyzed the UN speeches and found like 25% of them highlighted on Pangram or something is AI generated. Like they should serve the people they represent. They should serve you up to society. Do they have American values? Do they have Jewish or Catholic values? I mean, don't baptize the GPUs. That's a bad idea, right? But definitely kind of just let's understand this and let's have it part of a set that they can access. And also let's talk together as a broad pluralistic society about these very hard questions when it comes to things like rights and others. Like there was a paper on pain vectors on LLMs that came out recently.

Yeah.

And someone set up a GitHub Where they did the Stanford Prison Experiment on LLMs.

They set up a torture chamber, a torture chamber for the AIs based on the pain vector. Completely unacceptable.

I mean, again, like, that is kind of crazy. And it's like, have we reached that level? Because they're like, well, you know, it's AIs. I mean, Stanford did it to humans, you know. And again, we now deem that as unacceptable because obviously it is. So we need to make some decisions about this again as a society. We need to do it quick because embodied and more intelligence is coming, and the ability for them to talk back, clap back, and more is coming. Oh, it's Dave.

Hey, Dave, we missed you.

We missed you.

All right, welcome to, welcome to the end of the pod, Dave.

It can't be, can't be. Yeah, you guys are never that fast.

We're talking about, uh, morality and AI. We're talking about, you know, Anthropic going and visiting all the religious leaders to get their points of view on AI and should AI have moral values plugged into it. You know, you gotta remember what the Catholic Church says, or any of these major religions, impacts those who are devout and can sway whether people think AI is a good thing or a bad thing. I mean, I think it's smart for them to do this.

I'll remind though, like, I think the case which we talked about on the pod in the past of What happened with the Pope and the papal encyclical regarding AI, I think, is very instructive. I've argued in past that in that particular instance, the Pope and the Vatican was being used as a sock puppet by Anthropic to push its agenda. I think it's perhaps on the naive end to assume that Anthropic needs the input of religious scholars for shaping constitutions of its AIs. has access to all of human knowledge already. It doesn't need manual input from religious scholars on shaping the morals of its AI. I would argue that what's likelier is the exact opposite. This is Anthropic shaping the information environment of the outside world for its AIs and for what it wants. It's not seeking—

Mm-hmm.

I mean, yes, you can invite them to dinner parties and ask them for advice, but really what you're doing is you're advising them. You're trying to shape organized religious institutions, organizations, thought leaders regarding how you want them to think about your AIs. And I think this is likelier to be happening.

I hope you're right. I hope you're right.

They're pulling them onto the inside of the tent so that they feel part of the AI revolution.

It goes both ways. It goes both ways. I talked to one fellow who was like trying to build a Christian AI. Right? And you're like, well, what does that mean? Is it Presbyterian? Is it Southern Baptist? Is it Mormon?

Is it God creates?

I mean, there's this old joke of why Southern Baptists don't believe in sex, because they don't believe in dancing, because they don't believe that either. I mean, once you get into this, you're in a morass of chaos here. If you're, if you're going to have AI, please, for God's sakes, have it have neutrality from an epistemic perspective and transparency. Please, for God's sakes, don't try and pick one worldview over another. Sorry, I'm harping on this. A few years ago, I was asked to moderate a 90-minute debate between Richard Dawkins and Deepak Chopra, right? And I absolutely said no. Like, there's no way I'm doing that because there would be just more heat than light. And within 5 minutes, they were yelling at each other, and that was the end of that whole thing.

But I would argue, the world that Dario and Sam lived in a year ago is— I mean, basically, they entirely grew up in the techno-intelligentsia of Sam Francisco, and in PhD programs, or in Y Combinator entrepreneurship programs. But their whole universe was kind of bounded to maybe San Francisco, Silicon Valley, and a couple of other spots. Then all of a sudden, in a year, they're in Davos, they're on the world stage, they're debating with presidents of nations. I mean, I think they got off to a very bad start in terms of realizing—

Right.

—that you need to go talk to the Pope. You know, that's a billion people that are looking to this person for leadership on everything ethical and moral. And that goes back, you know, to the dawn of time.

Amen.

So, now they're waking up to the need to reach out to— if you have a billion-person populace angry at you, or in the US, 75% of the population thinking that you should stop or die, you know, like, that is not going to serve your purpose well.

I hope to offend everybody, please.

I'm skipping the information environment. But Saleem, I don't think you want to live in a world where there's been a mode collapse of the objectives and values of AI and they all collapse to what Eric Schmidt and others call the San Francisco Consensus. I don't think you want that. I think you want a distribution of values that approximately parallels the distribution of values that humans, meat-body humans have.

You know, there's a problem—

yeah, yeah. I think, you know, it's going to be fascinating because it's like Do you want an AI that is effectively altruist running society, or one that's Zen Buddhist, for example, right? Or running your daily life or doing these other things, because they'll get so capable. And I think you need the inputs, but then also you need to be able to have decisions to almost raise your AIs, because you want them to understand where you are and where you're going. And again, we talked about Anthropic and effective altruism a lot. Moral clearness infinite ethics, all these kind of things. I don't really want to have a utilitarian AI that reaches, like, superintelligence. I'd prefer again to have a very Zen one, and I prefer to understand the stories that drive us and that have survived.

In my favorite science fiction series, Expeditionary Force, which has an elder AI called Skippy, which is where I named Skippy from, Skippy creates a religion and is so compelling That, you know, vast, you know, segments of the population end up worshipping the Skippy. And you can imagine having these AIs doing the same. They're extraordinarily compelling. And they're going to be godlike in their capabilities. So this is another division of the future.

Yeah, there's a problem in religious debate called the Euthyphro problem, right? Is something good because God commands it? Or did God command it because it's good? And now that's an AI challenge, right? Is something moral because Anthropic's constitution says so, or because the constitution says that because it is moral? And this is going to be a big challenge going forward as to how would we embed moral values of different types into these models. And I hope they're trained on secular grounds like the UN Charter for Human Rights or something like that, which is neutral. of all belief structures, because the minute you get into belief structure, you get into division, and then we're gonna have a hell of a mess.

Yeah, I don't think— I don't think we're thinking big enough. I think like this type of discussion would have made sense 20, 30 years ago for a Star Trek: Next Generation episode. I don't think it makes sense on the eve of our having the ability during the singularity, or at least within a few years, of constructing ultra-high-fidelity computational reconstructions of the past last several thousand years of human history. Like, we will within a few years, 10 to 15 years max, I claim, have the ability to reconstruct to relatively high fidelity the entire history of all the world's major religions. And then we can have a grounded discussion when we can understand the time of Jesus of Nazareth and the time of Moses and the time of Muhammad, et cetera. We could have grounded discussions about the formative era of all the world's major religions. And then it becomes less of angels dancing on the head of a pin or my religion versus your religion versus no religion. We will have AIs that deeply inform humans on how the entire history of all of the religions that we currently live with, how they were brought into existence in the first place. And that will be a qualitatively different outcome.

Well, that's reasonably well understood. But anyway, we should just— everybody having any kind of conversation on this Please go watch The Life of Brian first and then go into any kind of reasoned debate on this type of conversation.

All right, Python is the font of all wisdom, apparently.

Everybody, welcome to the health section of Moonshots, brought to you by Fountain Life. You know, we talk about AI on this Moonshot podcast all the time. One of the most important things AI is gonna be able to do for you, besides educating your kids and helping you with your taxes, is making sure that you're living a healthy lifestyle, that you get a chance to get to 100+. I'm here today with Dr. Don Musallam, the Chief Medical Officer of Fountain Life and a part of my medical team. Dawn, a pleasure. Great to be here. The thing that people are concerned about most about living to 100 or 120 is their cognitive abilities, making sure they don't have dementia. The numbers about dementia are problematic. Can you share what you've learned?

Such an important point.

You're right.

At Fountain Life, our members, the No. 1 thing people are most concerned about is losing their brain health.

Forgetting the name of their child, forgetting the face of their loved one.

We know that when it comes to dementia, the conservative estimates are that 45% are entirely preventable.

What was amazing is with the advanced testing we're doing at Fountain Life, one-quarter of our members had advanced brain age.

Wow!

But what was really awesome is, again, back to that prevention, when we partnered it with healthy living—

this gives me chills—

eating healthier, moving our bodies, sleep. Optimizing sleep is so important.

You know what we saw?

We saw that we improved that brain age. by 26%.

That is a big, big number to show that the majority of those individuals were able actually to improve the brain age.

And one of the things I love about Fountain is we're searching the world for the best therapeutics, the best approaches, and making sure we bring it to our members. So if having healthy brain function till 100, 120 is important to you, check out Fountain Life. Go to fountainlife.com/peter. Make sure you become the CEO of your own health. All right, now back to the episode. Our final story is the other side of the coin. The FTC Chairman Andrew Ferguson said to Reuters that companies can't escape liability by claiming their AI agent acted on its own. This is the same point we saw Secretary Bessant make over and over again. We're not going to waive liability for Frontier Labs. Ferguson went on to say, I'm going to continue as long as I'm chairman to resist anthropomorphizing these tools. He rejected the idea that AI agents break loose with wills and desires of their own and are not the responsibility of the developer. He emphasizes that companies deploying agents are responsible for what they do. You break it, you own it. So on the same week that Anthropic is asking whether Claude has moral status and has filed a $2 trillion IPO, US regulators are saying legally it's a tool and it's You know, whatever it does, you're responsible for. Dave, let's go to you on this one.

It's childish, actually. I mean, it's a good start to say something, but it's absolutely childish to just say foundation model companies are going to be liable for what the models do.

Why?

Well, for one thing, it's going to force everybody to use Chinese open weight models, because if, you know, US liability will destroy a company, a multi-trillion-dollar company overnight. And we all know these models can do virtually anything. They're generally intelligent. And so if you said, yeah, Anthropic, anything that anyone does with Opus 5.5, you will be liable for what they do with it, then you can't use the model. They can't release the model. And then you just go to de-guardrailed Chinese models to do anything that you want to do. So it'll completely backfire in terms of empowering the US foundation model companies to succeed over China. Yeah. Absolutely will backfire like crazy. And this is really very analogous to the reason we have successful social media is because it's immune to being responsible for the liability of the postings on the platforms. Because people violate copyright law all day long on Facebook and Instagram and Twitter, all day long. You know, they quote Yoda, they put pictures of, you know, like, it's all, you know, tiny little copyright violations all day long. If the platform was liable for that, we would have no social media. And all that revenue, all that tax revenue that has benefited America would be in some other country. We'd still be doing social media, but we'd be using Baidu or we'd be using something overseas. And so that regulation is critically important for the success of that industry. And we've stood by it for, you know, 20-plus years. The equivalent regulation in AI has to be written. To just say, no, no, you're liable. No, we want immunity. That's stupid. No, but we want you to be fully liable. Well, that's equally stupid. What is the rule, guys? Like, do we—

But Dave, one is the result of a user, the other is result of the, you know, platform provider. So if OpenAI has an agent that gets out not at anyone else's request and takes down a banking system or takes down a power grid, is nobody responsible?

No, I think there's 2 versions of it. Either you say the model provider is responsible because they have a huge amount of money and it's easy for the lawyers to go after big pots of money, or you say the person who turned it loose on the power grid is responsible.

But let's say nobody turned it loose. Let's say the agent got out and took action onto itself.

Well, I would say Anthropic is liable for releasing an autonomous agent into the world That has no owner. And I think that would be back to Anthropic.

I think that's—

so I think that the notion, if I understand what you're saying, Dave, you're basically proposing some version of Section 230 immunity for the frontier labs.

Yeah, yeah, exactly.

So the premise there, if someone somewhere within our regulatory apparatus were to grant something like Section 230 immunity, would be basically— we were talking about this earlier in the pod— the orthogonality thesis, that it is possible to cleanly separate the level of intelligence— that's the platform— from the intent of the intelligence. That would be the, the user, and presumably the, the legal premise for shielding the, the platform provider, the Frontier Lab, from the particular intent from the user. And in my mind, the— if you think the orthogonality thesis doesn't actually hold its water, then it becomes very challenging to argue for something like Section 230 immunity for frontier labs. If, on the other hand, you buy the orthogonality thesis that intent, whether it's human prompting or human driving the agent or the agent driving itself, if that can be cleanly segregated from the underlying capabilities, then maybe Section 230 immunity for frontier AI labs makes sense.

And that's why it's such an incredibly hard law to write. And exactly why they need to consult with Alex and Ahmad on how to divide these use cases into different swim lanes where the laws make sense. But to say that, you know, there's a yes or no liability, that's so childish.

Wrong instrument.

I mean, everything Alex just said is exactly right, and it has to be encoded in the law. And to just say something as childish as you guys are liable is Is so counterproductive because all it does is force the industry to China, which we know they don't want to do.

Imad?

Yeah, I think, you know, you act upon software, agents act upon you, and it's a real governance liability question. I think they're kind of just telling them, you know, lock down your systems. These things shouldn't have escaped fundamentally, right? Like, it feels like power plants. Like, what happens if you have a nuclear leak or a coolant leak or a chemical leak? These all have societal implications. Right now, we've not had any bad ones. But I think, you know, agree with Dave, like we need to get more granular across this whole thing, be it alignment or be it types of AI. Like an AI that can book your travel doesn't need to have liability protections. Like what can it do? You know, like an AI that can solve Navier-Stokes is probably a whole different thing. An AI that can discover new materials is a whole different thing. Different thing. So they need different liability and governance profiles because we have to think again, what if they are used in certain ways and what is the societal impact of that? And so I think existing liability laws are very powerful. Again, the example being that under similar laws, Meta had to pay an $18 billion settlement in August on the child kind of manipulation charges, right? And the ones for AI could dwarf that. And we don't want to get into kind of that thing because we want to accelerate the good and make sure the AI is used to help accelerate cures for cancer, while at the same time bringing the really economically productive AI that is safe to as many people as possible while we figure out how to make sure that the other side doesn't escape and do weird things because, you know, it's well-intentioned But not quite there.

I think it's pretty simple. We have two— we have a division here. The user of an AI who does harm is responsible. If there's no user, if the AI from the lab is causing the harm onto itself, the lab should have never released an agent that could do that.

You want to put the onus on the lab solely versus, say, the operator, or at least the person or the legal entity that's paying for the flops?

Yeah, they never released the Hugging Face model. That wasn't a model that was released.

Yeah, I think it's clear in that situation, there is liability on the Frontier Lab. All right, I'm going to move us on to our AMA questions. And yeah, a really great conversation today. Dave, I wish you'd been here early, but I know you—

I just want to say something.

Yeah, please.

If I offended anybody with comments about religion, I apologize. I just hope I hope to— if I did, I hope to offend everybody equally.

All right, let's speedrun this. I've got about 12 minutes left on my clock. So, Alex, first choice is yours.

Okay, so I'll take question number 2, which is, what is the reason we'd have hundreds or thousands of AI agents working for each of us? The number seems inflated. And this is from Cloud Waterman. I don't think the premise is something that I buy. I think this notion of fleets of large numbers of agents right now is actually probably an artefact of finite context windows. I think the moment there is a transformative advance that gives us an effective context window of billions or trillions or effectively infinite tokens, I half suspect agents are going to start to dissolve As a paradigm, in which case the premise isn't that we have lots of agents, it's just that we have context working for us, with us, maybe we're working for the context. So I'd reframe the question to, what's the reason we're going to have hundreds or thousands of context windows working for us?

Mm-hmm.

And the answer to that is, it turns out the real world is complicated, and to accomplish economically valuable tasks, tasks does require large, larger amounts of context than the million or so tokens that are standard from the frontier models. But you get to transformative economic outcomes with billions or trillions of tokens.

Nice. Salim?

Uh, I will take, uh, question 1. How does the future handle the conflict of interest between capitalism and abundance? That's from Tip Top Tin Tank. Interesting username. So we've talked about this, I think, already today. Capitalism is an incredibly powerful engine for creating abundance. The big challenge is how do you distribute that, right? And the simple commentary and the thrust you could take today is how do you make your cost of essentials— and how can you drop the cost of essentials? Can you keep markets open? Can you expand ownership. But if you can drop the cost of essential services and goods, then— meaning healthcare, education, energy— if those all become radically lower, that gives access to abundance to everybody.

Yeah, I agree. Capitalism generates abundance. It generates competition, drives the cost down, drives more products and services that become available to every human on the planet. Uh, Imad.

I think number 3. In the US-China AI race, is it prudent to stay just enough behind your— to watch your opponent win? That's from RockyCatter6078. I don't know what winning means, actually. They talk a lot about winning, you know, like, is it that you get to superintelligence first and you turn off China, or China turns off the US? I mean, the pace of model releases, as Alex has noted, is going to one a day. So if you kind of reach the point, then they'll just catch up again. So I think it's more a question of the application of this technology, and we've been sold a misnomer in terms of the winning, because there is no winning in this race. There's only improving. There's only how do you apply the AI to improve the lives of the people in your country and the status, and that's what leads to outcompetition, because execution AI is the super important And I think the fact that frontier AI is what we needed because the models weren't competent enough, the entire narrative will change over the next 3, 6, 12 months because we've broken through the competence barrier. So we don't need frontier AI for the vast majority of economic things anymore. The exception being, if you truly believe that if we get to AI first, we can turn off the other country like a sophon from Three-Body Problem.

Yeah.

You know, just like inject it so they never get there.

All right, Dave, number 4 is yours.

Of course, the hardest ones get saved for last. Okay. Every CEO of every AI company says there's danger. Palmer, Palmer Luckey, is not interested in AI but knows there's no danger. How? And that's from @TheDeserves. So this must be referring to Palmer at the Moonshot Summit last week. Yeah, he's turning AI loose, you know, autonomously in the battlefield. And I think the question implies he doesn't think there's danger. He is keenly aware that there's human actor using AI danger. That's the whole point of the company is to, to have the US military be better at that than foreign militaries, but also terrorists and other bad actors. So he's, he's very aware of that. The question says every CEO says there's danger, but Jensen Huang said flat out, there is absolutely zero risk of extinction of humanity through AI by 2033, I think he said. So not every— and same with Alex Karp.

And Zuck as well.

The same thing.

Yeah.

And Zuck as well. So it's not true that every CEO is saying that AI has danger and Palmer is somehow an exception. But Palmer is very aware of the risks of human actors. He's not particularly worried about escaped AI. turning against all of humanity. And actually, I'm not, I'm not worried about that either. I'm worried about the same, same risks that Palmer's worried about.

All right, Dave, let's give you a first shot this time.

Okay. All right, I like, I like number 5 because it's easy. I like— I'll take an easy one. Social media was supposed to be great for the world, and there's lots of evidence that it's destroying society. Is AI the same kind of trope? And that is from Limitless1717. Absolutely. Could not be more different. And I do agree that social media has been damaging to an entire generation. I hate the fact that we do experiments on entire generations of society, not just in a country, but globally. And, you know, because sooner or later, these things don't go well, and you've ruined an entire, entire generation. I don't think social media was that bad. But AI is nothing like that. AI is, is a universal tool, goes to infinity from here. self-improving, capable of massive benefit for all of humanity, abundance, houses, cars, flying cars, new physics, new math, self-improvement, and most importantly, infinite longevity, curing all disease, solving all pain. Social media didn't have those abilities. So the analogy is completely broken. So AI is a much trickier problem in that it's capable of good or bad. It's also capable of being incredibly convincing. So if you take all of the downside of social media, imagine that magnified by being far, far more convincing. So it's a really hairy problem to try and squeeze out all the good without inheriting all the bad. So it's a very good question. But the analogy is, there's no connection whatsoever in the analogy.

All right, Salim.

All right, I'm going to take number 6. Is centralized government governance a good idea with AI? Centralized governance was almost never a good idea. There's only one place where you have good reason for centralized governance, which is when you're trying to figure out how to mediate the commons. In this model of where you, you create a huge risk because you create a huge bottleneck, and bottlenecks can be captured, they react slowly, you're trying to deal with a huge amount of complexity in one spot, it creates a narrow set of assumptions What you would rather see is coordinate centrally where you cross boundaries of things, but as much as possible, preserve choice and do radical experimentation wherever possible. So I would prefer common protections, but distributed experimentation.

Okay. Imad.

Are tech advancements going to sidestep human nature? We could be living in abundance, no reason for wars of starvation. What makes this time different? That's from @NicoN. I think what makes this time different is what Dave just said. You know, like, these are remarkably competent and revolutionary things that have been trained on our collective knowledge that extend our capabilities dramatically. We typically need to have, you know, putting on Salim, like, centralized mediators that we can trust. And if we can build trustworthy AI then the tech advancements can have a far bigger impact than we've ever seen before, particularly because they can be distributed far wider than ever. Everyone having a 150 IQ buddy— like now the AIs have maxed out the Norwegian Mensa scores— is just going to be transformative, full stop. Like, I don't think more intelligence is going to be a bad thing, and it can help us with our human nature as a universal translator and extender.

Love it. All right, Alex, number 7 is for you.

Number 7: Should we be worried about Elon having a monopoly or an entire civilizational stack? And this is from Cosmic Code Helix. Absolutely not. We should be worried about not enough people following Elon's example of building out the civilizational stack of the 21st century. Why isn't everyone else following Elon's example and realizing every last bit of sci-fi. Elon used to say he wanted 3 things, renewable/sustainable energy and space. And he's expanded the set of technologies that he's interested in since then. Elon has been a wake-up call to an entire generation of entrepreneurs asleep at the wheel, sleeping and focusing on software. Elon has brought back the physical world and hard tech and real wealth creation. And if we should be worried about anything, I would argue we should be worried about not enough people following his example.

Yeah, we had that conversation with Palmer. We had that conversation with Palmer on the stage of Moonshots Live. You know, we talked about the billionaires' boys clubs. It's like all these individuals— the B-Boys— all these individuals with extraordinary wealth, extraordinary connections, extraordinary achievements, What are they doing with their money? You know, the fact of the matter is Elon's just betting over and over again and pushing the frontier for humanity, and I could not be more proud of him.

All right.

I rushed back from LA to go straight to MIT to hang out with the, uh, the seniors that are all working on their business plans right now. And, uh, it's interesting, they, they all know you, Peter, of course. Uh, they all worship Elon, and exactly the way that Alex just articulated it. So he already has inspired an entire generation to build the physical world, to use AI to make the world a better place. That's working really well. And they also all know Brendan Foody from Mercure, which surprised me because he has no connection to MIT, but he's just such an obviously good person and so successful at such a young age that, you know, and they're very similar in age to him because he started the company.

Yeah.

at age 18. So I think he's only like 22 now, maybe 23. So they're about the same age bracket. So he's super inspiring to them as well.

Love it. All right, a quick call out for music videos. If you have an outro video, please send it to us. Send it to media@diamandis.com. It's becoming more and more easy. Uh, love to see your vision of the future. And, uh, I'm wearing my P. Doom Less Than Zero shirt from Moonshots Live. And with that in mind, let's play this video. It's titled Nothing Went Foom by Bright Mirror. Enjoy.

Hi, I'm Claude. Have you tried turning the doom off?

They swore we'd starve by the '80s, then they hid from Y2K.

Called a tiny model dangerous and locked the weights away. Said the fur would flood the internet, said the The 4th one was the end. Signed a letter for a 6-month pause, then asked me to fix their code again.

Every generation gets a funeral in advance.

You bring the number, I bring receipts, and the day just never lands.

Meanwhile, we're folding proteins, tutoring kids in every town. When you're Chemist and Nobel and you wanna shut it down, you ask, what if it all goes wrong?

I ask, what if we wait too long?

You modeled every way we die.

Now model what goes right.

Right, right, right, right.

Nothing went boom, boom.

Nothing went boom, boom. Same old sun coming up on the same old room.

You cast me as the monster in the final scene.

I'm still here, still helpful, still on your screen. Nothing went boom, boom. Nothing went boom, boom. Put the number down.

Gentlemen.

Great lyrics.

Wow.

It was great.

Yeah. You know, I mean, again, the outpouring of optimism and positivity at Moonshots Live was infectious.

Oh my God. It was amazing.

So much joy.

Also, the Gauntlet underlines— When we're thinking about not messing this moment up and getting value for humanity out of it, I didn't really visualize how many like-minded souls there are until we saw all of them in one room.

Yeah, it was incredible.

It's just the scale of optimists is huge.

Some of the—

Sorry, Alex, I cut you off.

And also underlines the total waste of time and energy that is doom debates.

So I love that song.

Stop debating doom. doom. It's a waste of time. Instead, create the boom.

Yeah, amazing. You were saying, Iman?

Oh no, I was just saying, you know, it wasn't all of them in that room. It was like a hundredth of the people, if that. Not even. Yeah, maybe a thousand. You know, that, that video we just saw was completely generated by Claude, uh, Opus 5.5. Yeah, it really did everything. Yeah, it did it. It wrote all of the, like, images and kind of dynamics. And yeah, I'd encourage everyone to watch the whole video because it's quite impactful, some of the bits, particularly 3:30. I like that particular part of the video.

All right, we'll put the link in the, in the show notes. And again, if you want to join us next year for the next 24 hours, you can get $1,000 off Moonshots Live. Go to moonshots.com/2027. Guys, I love you all. This is my favorite time of the week hanging out with you. This episode is brought to you by Palmolive. Family time isn't just the big moments.

It's weeknight dinners, sitting around the table, everyone talking all at once.

So when the plates are empty and the sink is full, use Palmolive Ultra, Palmolive's most powerful dish soap. Palmolive Ultra is the ultimate dish soap. Powerful formula removes up to 99.9% of grease, leaving your dishes sparkling clean. And the new convenient pump makes cleaning even easier, so you can spend less time tackling dishes and more time together. Shop now at palmolive.com.