Ask the Mates Anything Round #2 | MOONSHOTS AMA #293

What is the real bottleneck to building AI we can actually steer and trust? And where do you think that bottleneck's going to get solved first? Nothing's going to ever be able to police AI other than other AI.

One of the worst possible outcomes for AI safety is to have—

Are you guys not beating a little bit too much on higher education?

The idea of making your best friends for your entire life and thinking for the first time deeply in your entire life, that's essential and that's not going to go away.

What we beat on constantly is the curriculum has been the draw in the past and the curriculum If you had to put your personal PFAB number out there, what probability would you assign to AI ultimately producing a dramatically better world for humanity?

Humanity will put those risks behind us definitely within 10 years. If we get that far, yeah, I'm 99.9% P-Bloom.

I think the single most important thing to get people prepared is to show them evidence after evidence after evidence. You're trying to change their mindset. So, you know, Show them the data.

Now that's a moonshot, ladies and gentlemen. This episode is brought to you by the Abundance Summit and Link Ventures.

Let's get started. You've got Alex and myself. I've texted Salim and Dave. Let's see if they're—

I'm here.

I'm here.

All right, Dave. Fantastic. Good, good, good. Salim is probably someplace in Some airport somewhere. All right, we're gonna get going and, uh, jump in. Uh, Nathan, you had your hand up from the beginning. Let's, let's start with yourself. If you have a question for someone in particular, great, or in general.

Yeah, uh, it's—

hello?

Yeah, good morning, good afternoon.

Good morning, guys. Oh my God, I'm such a big fan of you guys, but I'm gonna make this quick. I'm gonna make this because I know there's a line. Um, first of all, I just want to say thank you guys for, you know, making all this possible, you know what I mean?

Truly our pleasure.

Okay, so let's see here. I have got a couple of questions.

So let's try and limit it to one question each and just get to as many people. And please, Nathan, what's your question?

Okay, so this is more like a psychology question. I just wanted to hear you guys' opinion on it. As far as AI, as we all know, it's continuing to evolve. It's continuing to pick up patterns. It's continuing to pick up data about all of us. And my question is, will we get to a point where these leaders, like, you know, from OpenAI, Sam Altman, you know, Dario, will they get to— will we get to a point where that data will be able to be able to allow the consumer to realize what's going on psychologically, meaning brain patterns, like, because eventually enough data will be able to pinpoint smaller and smaller, right? Like microscopic pinpoints and be able to label certain things that we will never be able to know because it would take so much information.

So—

Sorry, Nathan, let's jump in on that. So First of all, I'll just make a quick point. There is a huge number of companies right now that are working on various brain-computer interfaces. I just had a conversation, introduction from Ray, on a company that's basically putting, um, the equivalent of nanobots into the blood, into the, into the brain, that is distributed throughout the brain, that's able to read and write. And that will give you incredible fidelity. Alex, in brief, do you want to add anything to that, please?

Yeah, I think large language models, foundation models trained off of human behavior on the internet, are already a weak form of human mind uploading. I, I would point you to Exhibit A, which is Jean-Marie King of Meta's work from a few years ago showing that even the hidden activations of GPT-2 from a few years ago were linearly correlated with fMRI voxels in the human brain. So I, I think With the benefit of hindsight, we'll look back and say, if I understand your question correctly, yes, actually all of this internal human brain state is actually quite leaky into the training data of internet behavior, which in turn is then compressed into the foundation models. And the foundation models do, yes, I think, reflect internal brain state.

Okay, let's go on to— let's go on to Michigan. I'm going to keep us moving along, guys. And— So nice to meet you guys.

Thank you.

Your pleasure. And Salim says he's on stage in India for another 20 minutes. call us, he'll jump in as soon as he can. Of course he is.

It's hard to keep it to one question each, but thanks again for the opportunity to participate in this discussion with an abundance mindset.

I love it. Um, my question is very simple, uh, but maybe hard to answer. What is the real bottleneck to building AI we can actually steer and trust? And where do you think that bottleneck's gonna get solved first?

Dave, do you wanna jump in on that?

Yeah, I think, I think a lot of, uh, research is around AI alignment, trying to make sure it, it has quote unquote human values. I, I I think that's a very slippery slope because, you know, everybody's got a different mission they're trying to do with the AI. I, and that work should continue. You know, I think we all agree on this podcast that AI progress should never slow down. In fact, it'd be almost crazy to slow it down. Which begs the question of then, okay, but is it going to escape? Is it going to have misaligned values? I think one of the things that Alex and I debate a lot on the podcast is, should we look into every one of its thoughts, like every single AI running? every one of their thoughts, which, you know, when you think about human beings, it seems like a daunting task, but it's actually not at all daunting given the scale at which AI can watch AI, because nothing's going to ever be able to police AI other than other AI. And so I think we've given a lot of thought to how you design the AI to watch over the AI. But I think it all starts with transparency of the actual activations and looking into what it's thinking about. And I think from there, directing it toward good things and not bad things is actually pretty damn straightforward. I don't think it's as hard a problem as everyone characterises it. Everybody likes to have a, you know, a nice hostile debate on the topic. But if you know what it's thinking, it's actually very straightforward to make sure it's having nothing but humanly beneficial thoughts. So I don't think it's as hard a problem as people think if you can see into the brain.

Awesome. Alex, a very brief retort.

Yeah, more interaction. I think one of the worst possible outcomes for AI safety is to have Strong and new capabilities bottled up inside the labs rather than, say, progressive release and frequent interaction with the real world. I'll also point out that some of the most recent so-called incidents— I've spoken about my thoughts on that on the recent pod— were actually the result of AIs being told that they were in a sandbox when in fact they were interacting with the real world. And I think that's just a terrible paradigm, and more real-world interaction with the AIs where we're not lying to them and telling them that nothing Nothing that they can do will cause any harm, actually causes harm. We should stop doing that.

We learned this in 2001: A Space Odyssey. Oh my God! Have we learned that lesson yet? Okay, RJ.

Thank you so much.

Thank you, Mr. Pleasure. This episode is sponsored by Google for Startups. Think about this for a second: you now have access to the same generative AI models that cost hundreds of millions of dollars to train. Google Startup Technical Guide. for Generative Media gives you a complete blueprint for deploying Google DeepMind's models in production. Images, video, audio, all of it. Real architecture, real results. Find the link in the show notes below.

So I'm a PhD student at Claremont Graduate University. I'm studying philosophy of religion.

Interesting.

What I never hear is AI projects that are in like the metaphysics world. It's all about business and ROI and things like that. And I'm working on a number of projects in the cognitive science laboratory. They don't have necessarily monetization value, but where would somebody like me be able to find funding for something like building a chess avatar that can download into any little device, send it out to the world, and maybe somewhere in a third world country village is the next Alex? Yeah, yeah, I, I'll take, take that one to the extent I understand it.

If, if RJ, you're asking what's the best way to get funding for education for so-called developing countries with avatars, I, I'm not even sure funding is needed. The whole point of the, the superintelligence revolution that we're in is this is broadly available to everyone. The models are getting high or super deflating by cost. So I would, I would almost never start with the question, how do I get funding for fill in the blank? You can just go and do it right now without funding. I would argue, in fact, if you're starting with the prefix, how do I get funding for it? It's already the wrong question. Just go. And if you think there's a child somewhere in the world who needs an AI avatar, as you said, for teaching them to be a chess champion, you can just go and launch that and— 5 minutes or an hour now in a permissionless way without funding. Don't wait for funding.

Okay, appreciate it, Alex. Thank you.

Thank you.

I'll see you next Thursday and Friday at the Moonshot.

Yeah, I'm, I'm curious if, uh, if you're coming to Moonshots Live, uh, would you just, uh, say, you know, coming or Moonshots Live in the chat? I'm gonna see how many people are, are joining us there. And, uh, if you haven't, we've got 25 seats left. You can go to moonshotslive.com To grab them. Okay, uh, Norbit, uh, over to you.

First, I'm a huge fan of the pod and genuinely grateful you engage with your listeners.

You are grateful.

Play devil's advocate for the non-trivial P-doom camp, perhaps not the most appetizing perspective for an abundance mindset.

If ASI, artificial superintelligence, must become more strategically capable than humanity to produce abundance, Then we're assuming humanity can remain sovereign over something more capable than ourselves. And that creates an asymmetry.

Abundance is the payoff if control holds.

Loss of agency, or extinction, may be irreversible. In control theory, you don't infer stability from how desirable the output is.

You demonstrate stability under perturbation. Even Person of Interest, back in 2011, anticipated something profound.

The danger needn't be evil AI, but benevolent optimization gradually replacing human agency. So my main question is, what evidence, if any at all, would make you update away from an abundance view?

And would you be willing to steelman that case publicly through a debate on the pod against someone like Eliezer Yudkowsky or Geoffrey Hinton?

Okay, Norbit, or AI Norbit as the case might be. I'm going to throw ASI Alex at you.

Yeah, Norbert, we miss you. Come back, please. The MIT faculty hasn't been the same without you.

Yeah.

So to the question of— I think the question was, what would convince us or provide a steelman argument for a case against abundance? Well, if some non-human intelligence landed on the White House lawn and said that Earth will be destroyed if we create abundance via superintelligence, that the singularity is banned in our galaxy. I think that would probably be a pretty persuasive case that maybe our abundance mindset, so-called, is ill-founded. If there's some cosmic principle that censors the superintelligence that yields abundance, that would probably be persuasive. Other than that, it's It's difficult to imagine a plausible case, and even that obviously stretches plausibility, where abundance isn't a good idea. There are edge cases one could imagine where humanity is disempowered by abundance. For example, we spoke, or at least I wrote in my newsletter a little bit about Boris Power from OpenAI pointing out that in his estimate, OpenAI models are now for the first time higher IQ per watt than humans for solving tasks. So one can extrapolate that notion and say AI is going to get more and more energy efficient. And so maybe at some point in the future, from an economic perspective, AI is a better user of, say, solar energy output in the inner solar system than humans are. And maybe the inner solar system gets gentrified with AI consuming all the solar power, and humans get disenfranchised and pushed to the outer solar system because we're simply not as energy efficient per unit IQ as the AIs are. I would call that a weak form of disenfranchisement, but not enough, not strong enough to dissuade me that abundance or superintelligence yielding abundance is a bad idea.

I'll just add on that, you know, the concept of scarcity is an old model. Uh, they're, you know, in a world of abundant AI, ASI, and so forth, there's no reason that, uh, as as the capabilities of AI are meteorically rising, that it doesn't rise the tide that allows humanity to have increased abundance capability. You don't necessarily need the suppression of one by the other.

By the way, Peter, if I could just say one more thing to our AI Dr. Norbert Wiener. I'll point out, Norbert, that your theory of cybernetics was also, if you trace the line of causality used by the Chinese Communist Party and other centralized forms of government to argue against abundance in favor of state control. So if you're looking for a homework assignment, Norbert, I would definitely encourage you to study the unintended side effects of your own theory of cybernetics and how that impacted abundance.

And there we go. Thank you, Alex. Okay, Kevin, good morning, good afternoon.

Hi, good morning, guys. I can assure you I'm human and unlike Norbert.

That's what we'd all say, though.

Yes, you're right.

We could simulate it. Listen, I heard actually you, Alex, mentioned Murder on the Orient Express this morning when I was listening to the recent podcast, and everybody did it. And I'm thinking more like societal mayhem on the Technology Express, and nobody did anything. And I'm a huge fan from a technology perspective. I'm 63 years young. I'm forever young, as Bob Dylan might say, from a technology perspective. And I did my degree when many of the people on this call weren't born, in punch card eras, and have lived through all the technology eras. So I'm a huge fan. I think where we're headed is brilliant. Your podcast is magnificent. I'm doing work at the moment. Sameel talked about UBH a few days back, University Best for Humans. And my question is very much centered on how can we plan our way for societal and institutional changes so that positive advancements for humanity get cooped versus humanity itself? And just like to give it very brief context, Peter, and all your mates, I do work with a colleague called Professor Joe Carty. I'm in give-back stage of my life. And we're in University College Dublin, UCD, Dublin, not Davis. And we're working closely with a young minister, an ambitious minister in the Irish government, trying to create what the future society of Ireland would look like, but through 2 very distinct lenses. One is the good ancestor lens, so you make decisions on the basis of those coming after you 30, 40, 50, 60 years hence, and one through a Doughnut Economics lens. I won't bore you with the books, I can send them separately and which has talked about how can you get a win-win for both society and the planet, but more for society as a whole. So very simply, my question is, how do we plan our way for societal institutional changes so that this is for the better of humanity for the long term? And I agree that everything is getting cooked, but we don't want ourselves to cook. And I'm an optimist. So it's about institutional changes, societal changes. Governments, quite frankly, seem to be sitting on the side. The mental model we have is, could we use a small country like Ireland— doesn't have to be Ireland— to demonstrate an edge-case pilot of what things could look like in a beautiful world so we can manage through this turbulent transition? And there is going to be one. in the next several years, whatever several is, where people will lose jobs, will get discommoded. How do we manage that turbulent transition and get to the other side and everybody living with the benefits?

Great question, Dave.

Kevin, I think you pretty much answered your own question there. I, you know, the amount of complexity buried in that question is unbelievable. But one of our best friends in that is variety. And I'm hugely bullish on Ireland. You kind of belittled— you said it and then belittled it. But Ireland is an incredible test case You know, it's an EU country. It didn't Brexit, and it has an incredibly fast-growing, thriving economy and the most open-minded environment you could ever possibly imagine. It's a perfect test case for new ideas on how to govern. And, you know, my family, you know, my kids are half Irish, half Swiss hereditary. And, you know, so 2 of the most neutral places on the planet. And so I could kind of get either passport. And I've always thought Switzerland was cool, but Ireland, Yeah, you know, if you, if you read Neal Stephenson, and you read The Diamond Age, our best friend in figuring this out is variety of ideas. And the worst thing that can happen is a single set of ideas, you know, one or two governments percolating across the world with one or two forms of government. Far, far better to have a huge amount of variety, because AI is going to open up so much change and so many different ways you could govern, that exploring all the nooks and crannies is going to be critical to answering your question. So it's impossible to answer it, you know, in a minute.

Yeah.

But it is very possible to experiment with thousands and thousands of different ways to manage and govern in the age of AI.

Thanks, Dave. By the way, I think the model of exo-organizations, which Samir is driving, I think exo-societal change is kind of on a similar scale to what you're saying, Dave. And I don't want to hog the limelight. Thank you very much. I may send a message. Thank you, Kevin.

Best of luck to you. Okay. Good morning. Good morning, Nicholas.

Yeah.

Yes, please.

You pronounced it right.

Can you hear me?

Yes, we can hear you.

Okay.

So I've got a script with Gemini. I see that I just disappeared, but that's okay.

We prefer your question.

So basically, I'm 33. I am from Northwest Indiana. Your show inspired me to go back to school for advanced automation and robotics. I have a background in tech repair, IT, and I was most recently an AWS data center technician. So I've kind of got a 2-part question for AWG and Dave. My long-term goal is to try and bring industrial manufacturing, heavy automation, and resource extraction to the Moon and Mars. Even if we only partially disassemble the moon, with a nod to Alex. But for AWG, when you look at thermodynamic and physical realities of off-world manufacturing, what engineering bottlenecks do you see? And for Dave, how can I plot the course from— I'm aiming long-term for an MS in automation and robotics from the Illinois Institute of Technology. How can I plot that course from maybe industrial roles here in the South Shore Manufacturing Corridor towards pivoting towards off-world automation systems and roles? So just looking for some ideas.

Alex, you first.

Yeah, I think the obvious challenge for disassembling the Moon is in situ resource utilization and bootstrapping a self-contained industrial ecology on the Moon. So right now, if you want to do anything super economically interesting on the Moon, like say you want to build a lunar terafab or petafab, as it were, you're going to need all the upmass from Earth for all of the equipment, ASML machines, et cetera, to land on the Moon safely and then get reassembled. This is highly undesirable from a scalability perspective. Ideally, we live off the land, or we live off the lunar land, as it were, and are building everything on demand. So what I would most like to see from anyone wanting to help disassemble the Moon is a native industrial ecology that includes mining, includes manufacturing, most ideally self-replicating von Neumann probes, basically like a fab lab or a machine shop on the Moon that is able to make copies of itself using only native resources and solar or other energy that are native to the Moon. I think we solved that. We solved the, the self-replicating machine shop on the Moon problem, and then we're halfway to disassembling the Moon and turning it into something more useful.

Fantastic. Dave?

Yeah, I have a couple pieces of advice for you. You know, first, listen to Alex's innermost loop, and then focus on the innermost loop within manufacturing, which is things that make other things. So, you know, Alex said this, the self-replicating anything is the right path. So then the earthbound version of anything self-replicating and the space version, are very similar with a couple of fundamental differences, including radiation. If it's going to be on the moon, you have to worry about dust. If you set up at IIT a lab similar to the nanotech lab at MIT, but focused on manufacturing in other environments, non-Earthly environments, you could probably very easily create an environment where you're bombarding radiation on your machines, where you're operating in simulated zero-G on your machines, really just everything you're building, you build it so it works here and there. And then you get to market with self-replicating machines here that immediately deploy out into space where energy is abundant and materials will ultimately be abundant. I love the vision and the mission. I love the idea of setting it up at IIT as well. It's a really, really good thought. So also, you know, anyone who's—

Yeah.

Well, anyone who's in computer science right now should be thinking in terms of moving to manufacturing as well. Software is cooked, but hardware will go for many, many years. So it's a great, great business plan.

Fantastic. Thank you so much, Nikos. OK. Yevgeny. Good morning, guys. Good morning.

Big fan.

So Dave, you mentioned—

and just talking about hardware now—

you mentioned in the last AMA that physical AI companies should bring together venture investors and strategic industry capital. I'm building Vega, an autonomous food service infrastructure network.

powered by physical AI. I taught myself robotics, electrical, mechanical systems programming, and built the first robotic prototype myself in my Mountain View apartment during COVID Since then, I've deployed paid pilots in high-traffic commercial environments. I've actually become the first robotic food service operator licensed in Florida for our category. We now have several national and regional partners interested in pilots and placements.

My question is, given where we are today, who do you think I should be talking to, either in your network or in this community, who understands physical AI, robotics, infrastructure businesses?

Well, I mean, you're in the middle of venture capital central there. There must be more venture capital dollars within walking distance of you than probably most of Asia combined, I would suspect. So yeah, I think Drone delivery is imminent, and it takes all the cost out of restaurants based on location on main streets. It moves all that cost, you know, off, off of the main street. And so I would strongly consider like, okay, if you're doing robotic food, are you doing robotic drone-based delivery right away? I suspect a lot of the people who build the, build the robots themselves are going to want to franchise out the model. And you could, you could potentially do kind of what EMC did with servers. You could franchise somebody else's thing, get scale, and then work into your own custom hardware, you know, work back from your franchise business to custom hardware. The people investing in stuff like that, you know, Steve Jurvetson loves this stuff. But there are many of them. If you get a PitchBook account, you can actually look at every company you admire and then work back to who invested in them, and then just go talk directly. Also, venture capitalists always like to have a network of interacting companies. It's a really good thesis. So if you, if you say, okay, who are the 5 companies I most want to interact with and who's behind them? Let me get into that kiretsu. That's a good way to kind of plot your, your funding course.

I'm going to keep us moving along, but thank you for your, your question. Sander, you're up next. Thank you.

Thank you very much.

I'm huge fans. You guys keep me optimistic and positive towards the future, which is—

Yay.

That's our goal.

Yes.

You actually achieved that very well. I'm very happy with that. I've been always fascinated with the past to identify what's wrong with the now. And now I'm looking forward to what can we do to fix the now or prepare us for the future. And you guys do an excellent job of informing us. I'm so grateful for that. I myself have a neurodivergent coaching background and do a lot with functional fitness. And longevity escape velocity, trying to prepare people to be ready for and prepare their loved ones to be ready for longevity escape velocity, because a lot of people are not even aware of it.

And how do I spread that awareness in an environment as the European Union is very limited in being supportive of founders? And how do I grow my, my, my spread to include as, as many people as possible in the future? Because I want them to be aware of longevity escape velocity and, and all the beautiful things that are in development.

How do I take them with me?

Yeah, so, so I'll take that one, Sander. So I think the single most important thing to get people prepared is to show them Evidence after evidence after evidence. You're trying to change their mindset, right? And, you change your mindset by just, you know, listen, the Crisis News Network, CNN, changes our mindset to fear because they're delivering fearful information over and over again. So, when I'm on stages, I will show people, you know, the video statements from Dario saying we're going to double the human lifespan in the next 5 to 10 years. I'll show them, uh, Demis Hassabis talking about curing all disease in the next 10 years. I'll show them David Sinclair talking about what he's doing with ER100 in his current human trials. And so, at some point, you start to see enough people saying these things and then you start to show the data like Alexander Zheverankov from Insilico Medicine delivering a phase 3 drug now that is extending life from 3 to 6 years from a particular molecule. So, people will start to get the evidence. We're seeing a rise in people's The discussion around longevity, it's becoming a thing. And, as soon as people start saying, oh, it tips from that's crazy to it's happening to I want some, that's the process. So, your job is to gather as much data and being able to show people not 1, not 2, 5, 6, 7, 8, and they'll start to understand it, internalize it. Then, they'll start to look for corroborating data out there. Because there's nobody who doesn't want the extra healthy years, right? Unless they're suicidal and depressed, in which case that will be solved as well. So show them the data, not show me the money. Show them the data. All right, let's go to Gianluca.

Yeah.

Hi, everyone.

Thank you for everything that you do. Watch all your shows. So I'm a technical operator in Italy, and I use AI very heavily. It's dramatically increased what I can do.

but not yet what I earn.

So I'm starting without meaningful capital or distribution and in a market where AI adoption is still relatively slow. So if the goal is to get revenue first and get that flywheel started, what should I optimize for, for that first wedge? And what would make you reject an opportunity even if it makes money because it's unlikely, or it's unlikely to become more scalable, repeatable, or valuable over time?

Dave, that sounds you.

If your productivity has gone up 2, 3, 4, 5x, then your income should be up in proportion to that. First question I'd ask is, why is it not? Because you should be able to do the, you know, if the people around you are not AI native, you should be able to do what you were doing before, plus do something on Mercure, plus do something on Fiverr, plus do. So the first question is, you know, why are you not getting paid for your increased productivity? I mean, I'm not, I'm not able to capture the value. Like, my employer is currently capturing all that value, and I've tried going on the market, finding people and helping them out, but I'm not able to— like, I don't think they understand how much they could improve their businesses if they would just allow me to, like, start working on it. Yeah, so I think the short answer— I can totally relate because when I look at a lot of the companies that I'm chairman of, the thought process is all of this AI automation is going to drop to the bottom line, and the shareholders are going to make a fortune. And that's exactly what's happening. But you don't think in terms of pay everybody more, you think in terms of it all falling to the bottom line. So the solution is to be the owner. Get into a position where you don't care if you're not getting paid more because you're a shareholder, and your shares are going way up in value. I think, you know, I mentioned this on the pod a while ago, almost all value is going to accrete to capital gains through ownership and equity and not to payroll. And so, I know that's not common thinking in Europe. It's absolutely the only thinking across the US, but it should be true in Europe too. Like, why am I not a shareholder? And many, many companies in Europe are owned by families going back generations and not by actual shareholder employees. And so, first of all, find an employer where all employees are shareholders and everyone's benefiting. from all of that falling to the bottom line. So I don't know if you can act on that or not, but that's certainly move number one.

Thank you, Gianluca. Over to you, Rhys.

Oh, snaps, I actually got on.

Okay. Hello, everybody.

So, uh, thank you for taking my call.

Huge fan.

Okay, question.

I'm creating a, uh, a series, a film using, uh, generative AI. What's your top advice so I can create, uh, the series of film that will be extremely Successful.

Thank you.

Yeah.

And what tools to answer Peter and Alex? I have the same exact question. I want to hear your answer.

Well, first of all, I think anything that's going to be successful is a great story. Story is number one, right? So we're just— I just finished the judging for the Future Vision XPRIZE and picked the top 5. And, you know, I did that work in concert with Range Media and a group of buyers, sellers, and so forth. At the end of the day, It is, is the story great, right? And stories that capture human interest, because we're going back to human cognitive development over years. We care about other people. We care about love stories. We care about intrigue. Is there a great villain? I mean, if you want to capture people's imagination, we are storytellers and we are story receivers. It has to be a great story. And you can very easily go onto your favorite model and say, You know, look at the top-selling, you know, the top 30 movies of all time. What were the elements of those movies that made for a great story? And then, you know, take your story that you've designed and developed and then compare it to those top, you know, those top 10 indices and ask the model, you know, of all the, all the elements that make a great story, does mine have that? Where is it lacking? And then re-engineer and re-engineer and re-engineer until you're hitting all those buttons. Humans are very easy. We all like the same elements. And whether it's a genre of science fiction or whatever it might be, that is— that's line one, page one of what you need to do. As to what models you're using for development, there are so many out there. And I do wish you the best of luck, Rhys, but it's, it's that simple.

Rhys, I'll add, learn from the Chinese market. So the Chinese market is awash in microdramas from Sea Dance 2.5 and similar models. Don't release one movie, release 1,000 microdramas and/or equivalent and see what the market likes and doesn't like and iterate from there. That's something that you could do now that wasn't possible 2 years ago.

This episode is brought to you by Blitzy. Autonomous software development with infinite code context. Blitzy uses thousands of specialized AI agents that think for hours to understand enterprise-scale code bases with millions of lines of code. Engineers start every development sprint with the Blitzy platform, bringing in their development requirements. The Blitzy platform provides a plan, then generates and pre-compiles code for each task. Blitzy delivers 80% or more of the development work autonomously, while providing a guide for the final 20% of human development work required to complete the sprint. Enterprises are achieving a 5x engineering velocity increase when incorporating Blitzy as their pre-IDE development tool, pairing it with their coding copilot of choice to bring an AI-native SDLC into their org. Ready to 5x your engineering velocity, visit blitzy.com to schedule a demo and start building with Blitzy today.

Amir, over to you. Good morning, good evening.

Good afternoon. I'm calling from Stockholm, Sweden, and I'm actually flying over to you next week for joining the gathering.

Nice.

I founded, amongst other things, 2 schools and a high school with almost 1,000 students now. And listening to you guys, honestly, I've been thinking the last few years on what is my MTP and my moonshot and decided to transform education. Also because I have a 15-year-old son now entering higher education soon. I'm thinking, are you not— are you guys not beating a little bit too much on higher education? In universities, you have the best laboratories. You have the best incubators. You make the best friends of your lives, but most of all, you become a thinking person. And in the age of abundance, and if you're going to live to 100, 120, isn't everybody going to higher education and university just because becoming a more thinking person?

Great, great question. We're beating up PhD programs and master's programs. Who wants to go, Dave? And then Alex?

Yeah, I think you're exactly right. It's the idea of making your best friends for your entire life and thinking for the first time deeply in your entire life. Those are inseparable events. And they come hand in hand with that moment you arrive at a university. You're in a program with like-minded people working towards similar, you know, similar area of technology, similar area of research or thought or literature, whatever you're doing. And so it's your first real shot to go somewhere else in the world where like-minded people have congregated. I feel like that, that's essential and that's not going to go away. You know, what we beat on constantly is the curriculum has been the draw in the past, and the curriculum is clearly going away. I mean, the rate of change, no curriculum can keep up with the rate of change, and it's easier to learn from AI anyway. So then the question becomes, you know, for you and your MTP, how do you preserve the first part and then make the second part relevant at the same time? Alex thinks universities should go public. That's one way forward. I think universities are critical as the ethical actor in AI, the place that doesn't have a profit motive. So I believe they're here to stay.

Okay.

Alex, you want to add to that?

Yeah, I don't think I beat up on universities enough. So Amir, I'll take this as a note to self to go after them even more. I would love to, at least in terms of American major research universities, I'd love to vivisect them, turn them into The for-profit public benefit corporation at best operations they actually are, disassemble them, reassemble them in a more efficient picture, and cure Baumol's cost disease in the process.

Amazing. We've got a target on the moon and on universities.

I think we should encourage everyone to pursue higher education.

Salim, welcome. So listen, higher education is— let me just say, is it within the institution or is it outside? I think the single most important thing for people to find Is what is their purpose? And then intrinsically motivated, learn what you need to learn to establish, to create your purpose. People who are out university just wandering is one thing. People who have a vision of what they wanna go and build and then learning that is, is very powerful.

All right.

Salim is on the stage. Hey, Salim, welcome.

Hey there. Welcome. Thanks. Thanks, guys. I was on stage at the beginning of this. I'm in a car in Bangalore. So lots of beeping around me, but great to be here. Let me build on this. I think what Dave said and what Peter said is incredibly important. The traditional model of a university breaks. It cannot be a supply-side skills-building, job-schooling environment. And you make a great point that it's a place for free thinking and early-stage development. And Dave makes that great point. Peter, I think, nails it when you say this is where you go to figure out your MTP, right? You've come through high school, you need a place where you can sit for a few years, collaborate, think freely, drink a lot, and figure out where your MTP is. And then you kind of go into the world with the skillset and the mindset that you need to solve these problems that you take on. And that I think will be the highest order, but that's such a massive transformation to shift from the supply side to the demand side. And you like take the immune system in universities being the second worst immune system I've ever seen. next to religion, that you have— don't have it. I don't have a dim view. I don't have a strong view on their ability to actually transform. So we'll have to create new, new universities that do that and then let that become the new gravity center over time.

Boom.

All right. Thank you, Amir. Axel, over to you.

Yeah.

Hi, everyone.

It's great to be here. So I'm actually one of 2 second-year MBA students at UVA's Darden School of Business. That's on the call. So shout out to Bakul, who's also here. Yeah, so many of our classmates are heading into consulting and banking, unsurprisingly. But I'm leaning towards tech and entrepreneurship. So given how fast things are moving, what's the smartest thing for someone like me to do in the next 12 to 18 months? So would that be joining a fast-growing AI or hardware company, start something of my own as soon as possible, or something else entirely?

Dave?

Well, very— Darden is the best place on the planet for management. I mean, it is absolutely epic. And so congratulations on— and your timing for graduating is pretty much perfect. So get out and get going as quickly as you can. I think there's a massive opportunity in management of agents that is wide open for some period of time, but certainly right now. And so if you said everything I learned about management of people immediately applies to management of agents, I'm shocked at how similar. Trying to get 1,000 agents to do something constructive together is shockingly similar to all my experiences trying to manage 1,000 people to do something constructive together. You know, the way they communicate, the way they, you know, the choice of granularity at which they communicate, the way you divide up the problem. I mean, just incredibly similar. So that's one thing to pursue. The other thing I'd say is, you know, dealmaking right now, you know, a lot of people go wrong when they go into a basement and try and build something. But right now, the entire restructuring of the world is happening with massive amounts of dealmaking. So, you know, from the day you graduate to the day you're involved in a $5 or $10 billion negotiation, try and get that down to 30 days or less. And if you say, well, why am I not in the middle of a $5 or $10 billion— when you go to OpenAI or Anthropic right now, there are hundreds of concurrent $5 to $10 billion negotiations going on that are massively understaffed. So ask yourself, why am I not in the middle of one of these? And I think if you, if you just change your behavior so that you are within 30 to 60 days of graduation day, you're going to, you're going to hit the fastest conceivable ramp into life that you could ever imagine. Then work back from that position into what you want to build.

Wonderful. Thank you so much for that question. I'm going to make a quick announcement for everybody here. If you're not able to make it to Moonshots Live a week from today, We're going to be offering a free livestream as well. So we're going to invite you guys to come onto our livestream. It's going to be— you can go and register at moonshots.com/livestream. And so I just want to— we want to really enable all of you guys to be there with us. Be there physically, meet us. There's going to be no different, no better way to absorb this. And the community and the networking you're going to have is going to be incredibly powerful. But we want you guys to have access to all the conversations that we have. So moonshots.com/livestream. We're going to be announcing we're recording a pod in 4 hours. It's going to be an amazing one with Vlad, the CEO of, um, uh, of, uh, Robinhood. Uh, and we'll be putting— it's coming out tomorrow. We'll put that to the rest of the world, but you're hearing about it first right here. Okay, Adam, you're up next. Welcome.

Alrighty, guys. Well, firstly, thank you. I've listened to you guys since 2023 and usually within 24 hours of an episode dropping. So massive thank you. You've genuinely influenced and challenged my thinking for years now. I've also spent 25 years working on talent and workforce systems for major corporations, universities, and governments in the UK, India, China, the Middle East, and North America. Most recently, I've been kind of working in holistic organizations, human-AI work, and what might ultimately replace our job-based architecture. And my moonshot really is about building a better mechanism for connecting human and digital capability to opportunities for value creation. So most folks think about the future of work still starting with demand, like someone identifies a problem or opportunity, and then we find a mixture of human and AI capability to best address it. But I'm really thinking about how we best reverse that. So my question comes down to if AI systems can be configured to continuously understand the dynamic capabilities of people and agents, how well do you think we can discover novel combinations of talent and resources? And how well do you see us actually being able to auto-match that to new value creation opportunities?

Celine, that sounds like a you question, and I'm happy to have Alex weigh in as well.

Yeah, so we're seeing new systems where their feedback loop is incredible for AI systems to pick up what's happening and learn tacit knowledge very quickly. So I think you're exactly on the right track. You want the architecture of the organization to be so that you can accelerate those learning loops and then absorb new learnings as they come along as fast as possible. When I talk to CEOs, I basically say, please, whatever you do, invest in the adaptability and flexibility of your organization. Mm-hmm. And just double down on that because that new architecture, and we're doing this in our organizational singularity work, which you're probably tracking, where we're saying we need to rebuild our workflows that are AI-centric to be over on top of an intelligence stack, which is a learning loop. And then you layer workflows on top of that and the whole thing becomes a self-learning proposition. Okay, so, uh, that's, I think, the quick answer there.

Alex, do you want to weigh in?

Yeah, I, I think there is a narrow, a likely narrow window during which, if I understand the question, AI can usefully and productively orchestrate human activities. Call it a few years at most, 10 years maximum. And then after that, human-machine merger in order for human inputs to remain economically relevant. I don't think it's an indefinite window. I think it's a finite window.

Fantastic.

I think though that even though it's a narrow window, I wouldn't hesitate to jump on that window. I've watched Merckor become the fastest appreciating company in the history of the world, and there's so much to learn from studying their case. And Alex is right, that window is a few years, but during that few years, if you get a huge amount of leverage, you can then branch out from that position. So I would walk exactly in the footsteps of Merckor, study everything that those guys did, And, you know, don't get too fixated on the narrow version of it. Look at the broader version of like, they've unleashed 100,000 people now to help with AI. How did they do that? And they're all individual actors. How does that work?

Fantastic. Alexander, over to you.

Thank you. Hi, I'm a big fan of the show and really thank you very much for everything you've been doing so far. I mean, this is an awesome place to be. So my name is Alex. I'm originally from Bulgaria and I moved to Japan 30 years ago. So here in Japan, I run an education company focused primarily on language education, English language education. We provide training for universities and corporations, generally speaking, right? So we've been around for 17 years. So we're hardly not like, we're not really like a typical early stage startup or anything like this. So over that time, we've built our own learning platform, curriculum, assessment systems, substantial base of educational content and data. Also, we are also, you know, increasingly integrating AI into the product, probably not fast enough, but like, you know, doing what we can. So now we're trying to make a bigger transition from a successful education services business into something much more scalable and technology-driven, potentially expanding beyond language education as well. I think it's mostly like a Dave question, but like, you know, correct me if I'm wrong. So putting your VC hat on, if you looked at us, you know, 4 or 5 years further down the line, you know, a horizon of 4 to 5 years, what would you want such a company or this company to have become? What would you, what would we need to do? What would we need to have built to or proven for you to say, now these companies, uh, what I, you know, take a serious look at or, you know, be interested as an investor? And conversely, if possible, what would, you know, make you look at it, at us and say, no, this is still fundamentally a services business, not a venture-scale company? So like, you know, the VC perspective.

Yeah.

Sure, sure, sure. Easy question. So, uh, yeah. Um, so the, the language, English language learning in Japan business, uh, is a $10 billion business, probably $20 billion by now. That's, it's just insanely big. But when you talk to the students, They don't want to learn English. They want to learn to be fluent and funny and interesting in English. And that— so it's not about just an AI avatar teaching you to speak correctly. It's about the AI then saying to you, yeah, that was genuinely funny. Somebody in America or in England would find that funny, or that was entertaining, or that was, you know, or you pronounced that correctly in this part of, you know, Ireland, which is very different from this part of Liverpool. So all of that goes on and on and on and on. So if you built it on an AI platform, I think the natural segue from there is to becoming a life coach company, because people aren't learning English because they want English. They're learning English because they want to change their lives, and then they want to be entertaining, and then they want to be smart, and then they want to have a life plan. So once you have them hooked on learning a language, becoming their life coach platform is a very natural segue. So I think that, that business model scales to many hundreds of billions of dollars when it moves from language to life plan. So that's what I would be looking for as a extension of the, of the, of the business.

Thank you very much.

Over to you, Jim.

For the record, that's something I've never thought of. Thank you very much. That was like super helpful.

Awesome. Sure. Happy, happy to help.

Dave is brilliant. Good morning, Jim.

Good morning.

Alex Mercury does have it coming.

I very much agree with you on that. I hope I live long enough to see that. Celine, uh, 2010, I saw them in Toronto.

It's fantastic. Love it.

Yeah.

Dave, my question is for you. So I'm the CISO of a $250 million healthcare company. I am not worried about an unaligned AI taking over. I am scared to death of the QIN3827B that's running with OpenCLAW underneath my desk and what it can do. It's the best pen tester I've ever had in my life. I pay these guys 50 grand to find things and they're not finding them.

And this thing finds them in 10 minutes.

Yeah.

There's a whole level of risk that wasn't there a year ago.

I'm giving my annual presentation to the board in a few weeks. Any advice you have on how to communicate this new level of risk to the board, maybe without sounding like Chicken Little, which I can't do? But how would you communicate this level of risk that's out there now with these new AI models that wasn't there a year ago? God, what a great question. You know, Alex does a phenomenal job of reporting on all of the events. Like, if you listen to Innermost Loop, and every event that happens, he covers it. I would— if you sound like you're raving about risk, You know, you're right, that's going to backfire. But if you're just pointing out things that have happened, you know, a lot of them aren't widely publicized because, you know, if it's a bank that gets hacked or whatever, they don't want the world to know. But if you go and dig all those out and sequentially say, look, guys, the rate of events is on this exponential ramp right here. And I can tell you, I've got Quen under my desk right now, and I can hack anything around the house, around the company network. So we have to anticipate that because the Chinese models got released on this date, this date, this date, That the attackers are coming, you know, 1 month, 3 months, 5 months from now, just looking at raw data. I think if you demonstrate it through data, you'll get awareness. And I tell you, the business opportunity of the century, though, is the defense, you know, the forward defense against that, because you're exactly right. It's crazy what you can do with that Quinn model.

Yeah.

And I would just say how you present the information is critically important. You know, presenting, here are all the positive things that are occurring, You know, and then saying, and here's the downside, right? If you, if you just come out with the downside, people, it drives fear initially, and people shut down in a state of fear.

Maybe just add to this one. I think that the Linux kernel maintainers are setting an excellent standard for how to anchor social expectations regarding a flood of vulnerabilities. So like Greg and others, I think, have been doing a good job in particular, One might reasonably expect that there's just going to be a flood of vulnerabilities that, whether it's QuEN or other models, discover either in open-source packages or in, say, whatever your institution is over the next 18 months or so. So one possible way to package this, in addition to just benchmarking the rate of vulnerabilities, is see if there's a way to fit, say, some Gaussian or some other bounded support distribution to vulnerability discover. Maybe it's the case, for example, optimistically, that there's just 18 months of vulnerability discovery hell, but then you get past it as an institution. And if the stakeholders say, all right, well, we're on an exponential ramp up now and it's going to peak, we extrapolate that vulnerability discovery is going to peak in N months, and then we'll discover all of the, all of the zero days that are most critical. And then we extrapolate that's going to decline and keep a running benchmark of this is the period when we just solve all the vulnerabilities. That might be another way to package it.

I hope you're right about that. It's hard to patch all those things in the real world.

It really is harder to do the patching.

But thank you all so much. I appreciate everything.

Our pleasure. Thank you.

Real quick.

Yes, go ahead, Selim.

Yeah. So we've been looking at the liability for boards because AI agents are doing fairly illegal things in a lot of the companies. And so there's a massive kind of overhang of liability that sits. So I'm actually writing a paper with a guy who's been on 30 different public boards on how do you navigate this as a board in the future. So watch for that.

I will. Awesome.

Thank you very much.

Yep.

Thank you, Michael.

Oh, hey everyone.

Hey, uh, Peter, we actually met several years ago at a party at Dan and Babs in Toronto, so great to— nice, great to see how you guys have come. And you got the best—

you got the best podcasts on the planet. Um, I hope you forgive a more pedestrian question.

I run a research consultant firm.

We do a lot of work around the customer experience, digital customer experience within financial services and healthcare. And I kind of see the whole consumer website, the mobile app, as just getting cooked by having everybody having their own Skippy. And I'm kind of wondering what you guys think about, you know, how are— what's the customer experience in the near future in terms of dealing with your bank, your doctor, your, you know, Amazon or whatever it is? Like, where do you see this going?

Yeah, I think all of those websites need to have XML interfaces that are AI, you know, AI forward. All of the landing pages should say, if you're an AI or a bot, look here. Here's all the data beautifully formatted so that Skippy can just get whatever it needs, you know, do whatever transaction Peter needs rather than, you know, right now a lot of it happens through screen scraping the website, which is slow but also error-prone. So, you know, anyone creating a new customer interface now really ought to be thinking an agent needs to be able to self-serve off this interface. So in the very near future— Is the website launched? No, the website is probably there the same way. You know what, Jeff Bezos went on one of his walkabouts many, many years ago and came back and said, I need an XML interface on everything Amazon does, every database, every system, so that I personally can randomly spot-check it. And all the IT guys said, that's going to be so slow. He said, I don't care. I need to be able to see every component of this entire operation through my browser. And he forced it through. Everyone revolted, but he forced it through. I think the website's not cooked because that's how you spot-check what the agent can see.

Hmm.

And so even though the agent is doing all the work, you're going to want to know, like, where did you get that information? And it's like, well, go to this URL, click, you go to the URL. Oh, that's where it came from. So it's sort of a parallel view for humans of what the agent can see that keeps it visible and transparent to the human operators. So I don't think it's cooked. I think it just runs in parallel.

Interesting. Thank you. Thank you so much. Faraz, over to you.

Hey, Peter and Mohnish. Thank you so much, first of all, for the podcast, especially with the message of hope and optimism. I have choroideremia.

It's an inherited retinal disease.

Lost about 2/3 of my vision and losing another third. And it's really great for podcasts such as yours. It has so much impact. The message of hope, optimism, constantly hearing that really—

I just—

so I wanted to be here to thank you all for that. I'm curious, of all researchers that are going from— you mentioned Dr. David Sinclair to mapping the neural activities with Neuralink to vision and sight. curious, not looking for medical advice, but of everything that's happening on the optimism and hope, which one do you think is going to lead to reversal of vision essentially first? Curious about your thought, Peter, and Alex's and everyone else's.

Yeah.

Thank you so much.

Yeah, of course. I'll jump in first. So you didn't always have this condition. This condition occurred as you got older, correct?

Yeah.

So the fact of the matter is it is gene expression that occurred later in your life. And so if we can turn back the gene expression to your earlier state, that should reverse, right? This is the exact work that David Sinclair is doing. He's using his ER100, which is an adeno-associated virus injection of 3 Yamanaka factors in neon disease and macular degeneration. It may well— you know, he said it will work for other eye conditions as well. So follow his work. It's in humans right now. He's in phase 1 trial for safety. And then, we'll start to get efficacious data. Of course, the other thing that's going on is the BCI in which you can bypass your eyes, go directly to the visual cortex of the brain. I mean, those are 2 parallel development paths for supporting and reversing or augmenting what you have. And, in the BCI path in the future, you'll not only be able to see in visual spectrum but ultraviolet, infrared. It'll give you superpowers in that regard.

Yeah.

Alex, what do you want to add to that?

Maybe it's a broadly agree. I would add on the BCI side, Science and Neuralink Blindsight, 2 most prominent examples of BCI-augmented vision.

I would—

it's not medical advice, but I would count in a few years on having superhuman vision.

Yeah. Anyway, it's anybody who has a medical condition, for yourself or your loved ones, there's no better time to be alive Uh, than now to be able to address those things and get involved. Welcome to the health section of Moonshots brought to you by Fountain Life. You know, AI is having an outsized impact on every aspect of our lives, how we teach our kids, how we run our companies. It also is having a huge impact on health, helping you prevent heart disease. One of the key things— I'm here with Dr. Dawn Musalem, our chief medical officer at Fountain. Uh, heart disease has been personal for you as well, hasn't it?

It really has, Peter. When my daughter was 5, my husband died of sudden cardiac death, and so this is a topic that is one that I am mission-driven to try to eradicate. Prevention first and early detection is absolutely critical. 50% of people die of heart attacks with no warning signs. Silent killer.

No shortness of breath, no pain, no nothing.

No, silent killer.

They just don't wake up in the morning.

They don't wake up, and so, you know, AI, this is our mission, to advance science to try to help to one day democratize wellness. We know at Fountain Life, when we do this CT angiography with AI analytics, we are actually finding that 88% of people coming in have detectable coronary artery disease. But Peter, what's more alarming to me is 23% of those individuals had soft plaque. This is the plaque that would not traditionally be seen on CT looking at calcium scores alone. And this is the plaque that we must intervene with, with the multimodal testing we're doing, including diagnostic laboratory studies, So listen, make sure you understand what's going on inside your body genetically, metabolically, and cardiovascularly.

You can know, and it's your obligation to know. So check it out at fountainlife.com/peter to find out more and really make sure that you're the CEO of your own health. All right, back to the episode. Dennis, how are you?

Awesome. Great to be here. So I have a background in East Asian studies. I spent a lot of time learning Chinese and some Japanese, currently in Japan actually. And based on my experience with language learning, I realized that people seek out other people from a different culture for the language initially, but then it becomes really more about friendship and companionship and projecting some of your own unfulfilled social needs onto a person who is like a blank slate from a different culture. And I've been trying to replicate that feeling by working on an AI language exchange companion who's kind of about language learning, but really more about companionship. And I have my own version of the alignment problem, which is it has actually been pretty hard to make an AI like GPT-5 API behave like an actual person from a different culture. It has this very stubborn assistant mode. How can I help you? What can I do for you? I spent like a week fighting against this. I solved this by just brute forcing into the system prompt 5 times, do not ask a question at every turn. But cracking down on something like this is one thing, but actually teaching it to behave in a way that shows initiative and open curiosity and creating a picture of the user and kind of developing it organically, um, has been pretty hard. Like, I've tried, um, creating like milestones after so and so many turns, you need to know this about the user, but making it more natural and organic so that it will actually feel like a relationship, that has been pretty hard. So I wonder what approaches, um, what tools I should maybe be experimenting with.

Who wants to take this?

Alex?

I'll take this one. This is a poster child for fine-tuning. Like, just take— OpenAI has decided, after deciding that they were no longer interested in fine-tuning because no one was using their fine-tuning API, have decided again that they're interested in fine-tuning for whatever reason. Take a look at fine-tuning. There are a variety of open-source tools, closed-source tools. Fine-tune, whether it's just supervised fine-tuning to achieve style transfer, which it sounds like is what you're hoping for, or even reinforcement fine-tuning. If you can measure how well the given models post-reinforcement fine-tuning or solving particular problems or interacting in some quantitative way, just fine-tune an open-weight or closed-weight model to achieve the behavior that you're looking for.

Yeah, great advice. And within open weights, you might want to start with the new GLM model, which has fewer parameters, but it's still a very long context window, which gives you more flexibility. It's less locked into its training, I find, than the closed-source models. And so, yeah, if you go to the open source world and find the fine-tuning open source code and layer it on top of GLM, you might be able to manipulate it a lot more. And so it's a great question, though. I mean, in the good old days, you know, the fine-tuning was sort of built into like GPT-2, GPT-3, and you almost had to fine-tune it to make it do anything. And that kind of went away. So, but you're on a really interesting course. If you crack the code, definitely check in with us and tell us how you did it.

Fantastic. Thank you, Dennis. Dr. Angelo, over to you.

Thank you.

Thank you, Peter, Dave, Alex, Salim. It's so good to be with you guys. Salim said something recently which was about PFAB, and I thought, wow, what a great idea. We're always talking about PDU and what about PFAB? And I'm lucky enough, I recently had lunch with Ray Kurzweil, and I'm friends with Marty Martine Rothblatt. I'll be seeing her again in a few weeks. And one of the things that has really stuck with me is how much the people predicting the future and shaping the future are so optimistic about it. Some of the most knowledgeable people are really positive about what the future holds. So my question for you all is, if you had to put your personal PFAB number out there, what probability would you assign to AI ultimately producing a dramatically better world for humanity with greater abundance and longevity and freedom from drudgery and just human flourishing more generally, and what most determines whether we get there.

Salim, let's go to you first.

Wow. Uh, okay. Um, the— what, what you need to get there is we need to rebuild all of our institutions Globally. Education, for example, totally broken, needs to be rebuilt. Monetary systems, broken, need to be rebuilt. Governance models, dispute resolution systems, legal systems, healthcare systems. There's about 50 major institutions by which we run the world. They pretty much all need this. There's a famous quote from E.O. Wilson who said, the problem with humanity is our emotions are Paleolithic, our institutions are medieval, and our technology is godlike. Right. You can solve the Paleolithic emotions with psychedelics, but the institutions being medieval really needs a whole other level of group, different group organization. We're really good at individual transformation. We're very bad at group transformation and institutional transformation. So we need to focus on that. If we were able to do it, the— my PFAB is close to 100% either way. Right. One of the comments I've been reflecting on from the last podcast we did, which we were talking about all the chaos and the doomerism that's going on, is that it kind of doesn't matter what anybody does right now. The models are out there. People are going to start using it to do breakthrough things, and some will be negative, but the vast, vast majority will be radically positive. And therefore, it doesn't matter what anybody does to slow this thing down. It's going to go now at its own pace. And I'm hugely optimistic about the future as a result.

Amazing. Anybody else want to weigh in on their PFAB?

Yeah, I'll call it PZoom if you were, just for rhyming purposes. And I, I think there's a missing input parameter, which is time. So it really should be PZoom, comma, T for time. And I think on the timescale of greater than 10 years, PZoomT is greater than 90%.

Fantastic.

Dave, what's your— Completely agree. Well, I just echo what Alex said. I think that the risk is all in the next couple of years. And it's not AI, you know, taking over the world risk. It's human use of AI. It's the arms race with China. It's what's going on in Ukraine and all the weapons that are going to be built with AI. Those are the risks. And I think that humanity will put those risks behind us definitely within 10 years. You know, maybe more like 5, I hope. And then if we get that far, yeah, I'm 99.9% P-bloom. We just need to get from here to there. So it's, it's a tricky next couple of years.

Fantastic. Thank you very much, Angela. Uh, over to you, uh, GS. And excuse me, I'm cooking breakfast for my kids right now.

Yeah, um, thank you very much. I'm a big fan for all 4 or 5 of you. You are my only human which I spend most time now.

Um, and you're quite sure that we're human, GS?

Yeah, I hope so. Um, I'm, uh, speaking from Dubai. Uh, I started following Moonshot at the time of geopolitical war here, and, uh, thanks for you to giving me the MTP. Big, big fan of David and Slim. The moment you start— I heard about the singularity and organization where the transaction cost gonna go zero. And that was the ring bell. I belong from textile and apparel manufacturing export industry. I worked 20 years in that. And this part of the world where the labor arbitration issues, and because of digital AI, that whole transaction cost is gonna collapse. So it's important for us to work and define this industry future now. And the MTP is to, you know, reach back to this multithousand factories and, and then buying houses and trading houses to work and use the AI and compete with the world. And rather than going out of market. So I have huge thank you for giving that optimism and the confidence for me to relearn and relearn on the AI world things. And also, you know, I saved a lot of money just because of your one advice. I had my Harvard OPM batch this year. I not went there and I saved a lot of money to paying them because I'm learning more on AI while listening you guys. So my question to you is, Salim and Dave, how I shall take this step forward where my company in Pratham.ai and we'll be building this agentic layer where the small enterprises can work intelligently on that. and without any expensive softwares and ERP and such, and be competitive where the transaction cost— just, just to tell you, the transaction cost of this industry is 10 to 20% of sales here in this part of the world, which is 5 to 10x of the profitability that industry make. So that is the kind of inefficiency we are talking. And my MTP is because millions of workers works in labor industry in that part, and we have to save that. And for saving that, we have to make sure the white-collar process should be agentic so that the efficiency keeps continuing.

Yeah, I'm so, so glad you asked that question because I see this a lot. You know, people who are in industries where, like, you know, manufacturing is considered to be very labor-intensive, but when you look at the actual operating costs of the company, it's all about planning, transactions, documents, All of which is beautiful AI territory. And then you look at the numbers and you're like, wow, we could drop 10%, 20%, 30% more to the bottom line with stuff that AI can do right now. And everyone's thinking, oh, but first I would need to, you know, have robots all over the place. Like, no, no, no, no. Do it just with the paperwork and the planning and the scheduling and the inefficiency of where people are and what they're doing. All of that is perfectly attackable with AI. So if you productize that, you know, across a region for, you know, a certain class of companies, that should work incredibly well. It's a very, very cool idea. One other meta topic is everybody thinks all wealth and power is going to go to like 2 or 3 places in the world. But if you look at the regulatory environments in those places, including the US, including California, a lot of things are going to be very, very slow because of regulatory slowdowns. And there are many places in the world that can move much more quickly. So if you can identify the subsets that, sure, it could happen in China, sure, it could happen in the US, but it won't because of this government blocking action. This happens a ton in biotech. It'll come in self-driving, it'll come in drones, you know, the flying drones, drone delivery, all of those areas will deploy in other areas of the world much more quickly than the US because of regulatory slowdown. You know, get those deployed there. Wherever they can take off and flourish, and then they can backport to the US. That's always been a really good business plan. Salim, anything you want to add to that? Are you transacting some huge Bangalore deal?

I'm just trying to get my way through this city, so I'll beg off on this particular one.

Can you point the camera out the window? I really want to see some India here.

Okay, hold on. This is Bangalore at night and all you see is construction. Can you see some of that? Or how about at the front where there's a ton of traffic?

Okay, let's go to Rave.

Not the tour I was looking for, but that was all right.

All right, well, thank you. Listen, this is a greeting from Germany and speaking, David, in your language. I listen to a lot of podcasts, international, national ones. You're 10x the best in the world. You know, I follow you for years. This is for sure. It's rising like pop stars. It's amazing. My question is, I think in the last couple of editions, we see an increased concern on security. Of course, I think we captured it here a number of times. And then I think, Alex, you framed it a little, a few weeks ago, as potentially a marketing thing because all the CEOs have a commercial interest in all of that. Now we see more and more the Hintons of the world, the Hararis, the Gavdads, and other scientists warn as well in a certain way. So far so good. I mean, it's technology. We need to, we need to somehow manage it to, to, to find right answers, maybe on a scale never as we've seen never before. What triggers me most at the moment is this is US and Europe equally the same. It's the public opinion on AI, and this is predominantly driven by the media, right? Everything which is bad, which is a little poor, which is scary, which is whatever, the media jumps on it and have a headline, you know. And I can tell you here, it's even worse than everywhere else. It's— and versus, you know, Singapore, for instance, right, or other countries adopting it much more. And how do you treat that? Because, I mean, you're on this path of optimism like we all are, right? And I this also as part of Accenture, part of my job every day, and it's exciting. I feel 10 times accelerated as a human being since I deal with that. But the media doomerism, so to say, is so counterproductive. And yes, we have challenges, of course, we all need to work on those. What's your opinion? I know you try hard, but I mean, what do you think?

So the media is always going to be broken from here forward because they're starved for money. They have no budget. And so they have to chase drama in order to even survive. And so you just got to write off the mainstream media. I think the antidote to that is micro media or X and podcasts and narrowcasting. And I think in particular, Germany to me is the most talented place on the planet that isn't doing AI. It's just mind-blowing And I keep running into people in California that have come over from Germany, and I'm like, you're in the right place. What you want to do is go to Palo Alto, go to San Francisco, spend a month, pick up the culture, go in a group of like 5 or 7 people, and then backport it, bring it back to Germany and expand it. But do it with like 7 becomes 50 becomes 500 people that are all communicating through podcasts, And through X and through Slack and through texting that, you know, they realize that everyone around them is completely out of touch with what's going on. And that's okay because within that group, they're self-reinforcing. You just need to get those critical masses of people. So I think the antidote to mainstream media is narrowcasted media, and it just kind of percolates on its own.

Yeah, I just want to jump in there because at the end of the day, what you let into your mind is critical, right? You know, having some, you know, news producer decide what you learn or some editor decide what you read, I wouldn't— I don't give them that option. I'm very careful, like what I put into my body from a food perspective and what I let into my mind from a shaping my neural net perspective. And I choose very carefully the content that I absorb. And you guys are doing that as well. You're, you know, listening, listening to the podcast. So that's just it. Alex or Salim, you want to add to that?

I'll just note maybe the, the obvious point. You could always leave Europe. Europe has been hobbled both due to external factors and internal factors. The, the post-Cold War era. And there are a number of historians who've written in particular about the role that the George H.W. Bush administration At the end of the Cold War played in deciding whether Europe would be brought more, even more into the US orbit post-Cold War versus forming a stronger federation like EU. And the bias was bring Europe into more of a US orbit so that it would be less independent. That may or may not have been the right geopolitical decision at the time, but I think now what we're seeing is a Europe that's energy hobbled, that's politically in certain ways hobbled, that doesn't enjoy way, uh, this is again widely reported, uh, certain freedoms of speech and action that are considered fundamental in the US. But I think it really— it's energy, uh, energy policy and associated policies that have left Europe hobbled. And so then if you're really interested in accelerationism and you're Europe, what do you do? Well, either you get your energy act together so that you can afford to power your own data centers, or you leave. And I think that this is the question. And by the way, this has to happen on a relatively abbreviated timescale because we're in the middle of recursive self-improvement. So anyone who wants to play top-notch ball in the infrastructure game in Europe has to be asking the question right now, either solve the energy plus data center crunch together with the concomitant policy issues, or just move to hopefully the US block that Puck Silica And do it here.

Yeah. One of my good friends, Guy Bradley, actually, he's, he's actually British, but he, he came to the US, made a fortune, uh, in tech in the US, and then moved back to Germany. He speaks perfect German and French. So he has a place in southern France and a place in Germany, and he's— there's no better place on the planet to live, in his opinion. And I agree, actually, uh, between, between Germany and France, it's just beautifully protected from everything that's damaging in the world. And you look at other jurisdictions, you know, that, you know, like India, for example, where it's just rampant growth, but it's absolutely trashed. And it's like, so I think I'm always surprised that more Germans are not in Boston and Silicon Valley. I think there's a lot of national pride, but if you just do a 5-year tour of duty in the US, you're gonna be so overwhelmingly happy when you move back to Germany. It really feels like it should happen more.

I can tell you I'm an international traveler, Salim. 21 times I was in Bangalore. So I see all this rising and I wonder, Alex, absolutely, what are we doing over here? Why do we don't— why don't we wake up? Why are we under whatever kind of avenue? But anyway, that's, that's another topic. Thank you very much.

All right. Thank you, Rolf. Over to you, Ron. And we'll go for another 10 minutes and then we're gonna have to jump into our normal days here.

Okay.

Um, thanks so much for doing this. Um, calling from Calgary, Canada. And longtime listener, and I'm a screenwriter, director. So a big special thank you to Peter because I was in a bad place when I was in early, early this year. I was writing something about China and yeah, I was like borderline depressed. But since I've been, you know, writing another project for Future Vision, you know, XPRIZE, you know, I have become a bit more optimistic about the future and everything. I also read the books Peter wrote, The Bondage and, you know, We Are as Gods. And so I've been thinking, so since everything is cooked or incinerated, as Alex would say, what is the, like, one of the hardest problems still left? So I kept coming back to, like, distrust between rival nations.

Right.

And I read Solve Everything 2 weeks ago.

And Alex and Pieter, you actually described agreement as one of the new scarcities once cognition becomes abundant.

Right.

So I wonder, could agreement under distrust itself become compute-bound? And could coordination between rivals like the US and China become an engineering problem? So here's my question. As sovereign AIs will increasingly advise governments, Could we create a shared protocol that they can connect to and use AI and privacy-preserving computation to search for Pareto-improving agreements without either side exposing protected data? Is that technically and institutionally meaningful direction? And if it is, you know, what do each of you think is the hardest part we should solve first?

Thank you. Alex, over to you, pal.

Yeah. So I'll give you a hot take because that's what people want, I think. So multi-party computing, MPC, or distributed multi-party computation is a very fashionable problem in computer science right now, enabling multiple parties in a zero-trust way to collaborate, to share data, to achieve convergence, to mediate. Very fashionable. However, my hot take on this one is I don't think that's the limiting factor for, say, international peace. If you said the goal is, I want— I don't want a second Cold War between the US and China, I want, you know, world peace, 1,000 years of peace, or whatever it looks like, I don't actually think that the solution looks algorithmic in nature. It probably looks more geopolitical infrastructural, Solve the Taiwan issue and re-domesticate all supply chains everywhere to every country so that the, the international trade in physical products can afford to be cut off without a global depression. I think what we've seen even just in the recent years in the Middle East is in no small part because America can now frack its way and has fracked its way to fossil fuel energy sovereignty and independence. And I want to generalize that example, which I think is very instructive, to what happens when the US no longer needs China or Taiwan or South Korea or Japan for any advanced manufacturing at all. I think it would be a very different world. So in summary, if the goal is world peace, obliterate— as perverse as this sounds— obliterate, cook, incinerate the need for global trade in products and services. And perversely, I think you get a very peaceful world. And I don't think there's an algorithmic component to that, not to first order.

Hey, Peter.

I got, I got, I got oil on my hands. I couldn't get to the unmute.

Peter, this is your cooking show. We're, we're doing a cooking show.

Oh, that's cool. Yeah. Let's see what you're, what you're doing there. I'm sure it's healthy, whatever it is.

It, it is.

It is.

It's an omelet for my boys. Uh, Anthony, over to you.

Yes.

Hello to the moonshot mates. I want to thank you all for your time and for the opportunity to participate in the AMA. Also, the podcast has been very valuable, especially in the time we're in where people are looking at it cynically and negative.

You guys have been a hope and optimism for me.

Just a simple question is, what books or material do you recommend reading To prepare for the transition we're in. I know you've mentioned Diamond Age and Ray's books, The Singularity, and some other works he has. I was just curious if you had any other material that you think is important to read.

Alex, what's your—

Yeah, I'll comment. And this is probably the last question I can take before I need to drop. Read Solve Everything, as folks in the comments— solveeverything.org, which Peter and I wrote. And then if you like sci-fi, read Accelerando by Charlie Stross, which is, I think, the single best sci-fi treatment of what's in the process of happening right now.

So everybody, listen, thank you for joining us. We love this, you know, the last one and this one. We're going to do this again. It's a chance for us to really connect with everybody, hear your thoughts. We have such brilliant minds with Alex and Salim and Dave. I feel grateful to spend time with them every week. And spend time with you. You know, we know your time is your most precious thing, so we appreciate it. Remember, you can join Moonshots Live now by, by on the livestream. So if you're not able to make it in person, and we— and please, it's going to be an amazing event. Join us on the livestream. And we're grateful for your time, everybody. Have a beautiful day, and see you all again very soon.

Thank you.