Why AGI Is Close but Not Here Yet | Ray Kurzweil | EP #261

We just saw a story where Demis Hassabis, who you know, said 50/50 whether we need another breakthrough to get to AGI. What do you think?

Well, I think we need two things. So we've made a 75,000 million trillion fold increase over this 75 years, but AGI will happen by 2029. Large language models have only been effective for the last 6 months. We're being really affected by the exponential growth. A year ago, large language models were okay. Now they're really very effective, and we're really going to be able to feel that in the future.

If you could send a message back in time to the 1960s or 1970s for how to avoid plateaus and just speed up progress toward the singularity, what message would you send back in time?

I think we have to consider Now that's a moonshot, ladies and gentlemen. It's a pleasure to invite everybody to an afternoon with an extraordinary man, Ray Kurzweil, with my moonshot mates, my co-author Steven. So, Ray, you've been a mentor, a business partner, a co-author. For me personally and just an incredible guide for many of us. Ray Kurzweil is called the relentless genius by the Wall Street Journal, the ultimate thinking machine by Forbes, the rightful heir to Thomas Edison by Inc. magazine. PBS named him as one of the 16 revolutionaries who made America. He invented the first CCD flatbed scanner, the first print-to-speech reading machine for the blind, the first text-to-speech synthesizer, and the first music synthesizer capable of recreating a grand piano. He's a National Inventors Hall of Fame inductee, a National Medal of Technology recipient, a Grammy Award winner. He holds 21 honorary doctorates and has been honored by 3 US presidents. He's authored 5 national bestsellers, including The Singularity is Near, and how to create a mind. He has proposed the pattern— he proposed the concept of pattern recognition theory of mind, arguing that human neocortex is composed of roughly 300 million hierarchical pattern processors. That theory became his engineering blueprint. In 2012, he got his first job as the director of engineering at Google. With a singular mission: teach machines to understand human language, not just match keywords, but grasp meaning and context. His team helped build the Knowledge Graph and advanced semantic search so that when you typed "apple," Google finally understood whether you meant the fruit or tech company. His arrival helped trigger Google's massive AI talent grab. Shortly after he joined, Google acquired DeepMind, brought on Geoffrey Hinton, and expanded Google Brain, the research team that brought the transformer. His team advanced hierarchical deep learning and natural language understanding, helped shift AI from a niche academic pursuit to the core engine of the world's most powerful information company. He didn't build a digital mind. He helped build the foundation for machines that can finally talk back to us. He's made 147 predictions with an 86% accuracy rate. And his longstanding prediction that AI would reach human-level intelligence by 2029, well, we're here to discuss that. Ladies and gentlemen, please welcome Ray Kurzweil to the stage. Alright, Ray, center seat here, my friend.

Okay.

It's been quite the journey.

Yeah, how long has it been?

Oh God, we met, I think Martine Rothblatt first introduced us.

Okay, so it's about 20 years.

Yeah, well, let's see, what year is it? 2009, guys.

2009. Okay, well, the journalist here is telling the story here. Well, no, it was before that because we launched Singularity in 2009. I think we met when 2007— and The Singularity is Near came out in 2005, right? I think so, 2005. I remember I took The Singularity is Near, which is quite a thick book. I took it backpacking in Chile and I read the book making notes in the margins. And I had started with Bob Richards and Todd Hawley something called the International Space University back in 1987. We had the founding conference. And when I read your book, it changed my world. How many folks here in the room did the Singularity is Near change your world, right? I mean, amazing. And I said, there needs to be a university that teaches this stuff. Because all universities, you go down a very narrow niche. You become a hyper super specialist. And no place could you learn a broad version. I pitched you over a lunch, and then off to the races.

It was a dinner, wasn't it? Yeah, it was a dinner. It's true. Yeah. Can you hear me?

Yeah.

Yeah.

Yeah.

Well, I said yes right away because I make important decisions. Very quickly, so.

Yeah, it was. And we had our founding conference at which Saleem attended.

I got invited there by NASA. I'd set up a relationship between Yahoo and NASA. And somehow I'd never heard of the Singularity or X Prize or any of that. Walked in, top of my head lifted off, and I asked a few too many questions.

Yeah, and we made him our executive director. Little to his knowledge. Yes, yes.

I remember having a board meeting. He said, hey, come to the board meeting tomorrow morning. I was like, oh, and you'd ask me how much spare time do I have, and I said I've got a day, day and a half a week. I'm building a startup, as is needed in Silicon Valley. And you said to the board, all right, we have our inaugural board meeting. We've formed the board. We need an executive director. You've all read Salim's bio. He's agreed to do it for 50% of his time. And Ray said, I second the motion. Boom, all of a sudden it's ratified. Hanging up the phone and my wife said, how was the phone call? And I said, I think I'm a dean. I don't know how that happened.

So there was that. That's typical for startups.

There you go. Yeah. So empty seat, warm bum.

By the way, I know that you're busy and sometimes these episodes run long and you don't have time to listen to the whole episode, or if on occasion you miss an episode, I now put out a Moonshot summary on Substack, which includes a link to all the stories that we cover. The weekly recap covers what I and the mates had to say, what we think is most important, and what we're most excited about.

And it's free.

You can subscribe at diamandis.com/metatrends. That's diamandis.com/metatrends. All right, now back to the episode. So, Ray, one of the questions I'd love to ask, uh, we just saw a story where Demis Hassabis, who you know, uh, said he doesn't that 50/50 whether we need another breakthrough, a fundamental breakthrough in AI to get to AGI. What do you think?

Well, I think we need two things for this to understand physics. Like it doesn't really understand physics. It can infer that from the wording, but it really doesn't understand how different types of things would interact. Google has announced a project to do that. That I think will take, I think, till 2029. Then robotics is behind large language models. Large language models can basically understand everything But I've got to clean up after dinner. That's my assignment from my wife. Robotics doesn't understand that. Like this one, I need to put something in the refrigerator. This one needs to be washed away. Everything's a little bit different. To actually understand that and actually be able to do that, we don't have robotics that can do that at any price. It also needs to be made less expensive. People can't spend $100,000 to have a robot clean up after dinner. I think that will come about 2029. It's not there today. Those two things I think need some additional work, but we know what needs to be done.

Let's go around the horn here with the mates and then we're going to be opening up for yourselves in a little bit. Dave, you want to kick us off?

Well, I got to tell you, your very first book was such a life-changing moment for me to read it.

Spiritual Machines, which one?

It was the first one. It was Singularity is Near, I think it was the title of it. It's the one where he invented the term singularity, which, you know, a lot of people in AI had been thinking about for a long, long time, but nobody had crystallized it into a term. And, you know, now that the topic of are we in the singularity is going to come up constantly until it's in the rearview mirror. But it was world-changing for me because I had read a lot of analysis from Danny Ellis. I'm sure you know Danny, who built the Connection Machine, trying to predict how much compute will it take for us to crack OpenAI. And it's not an easy problem at all. And, you know, because you could simulate maybe a million parameter neural net at the time, and then, you know, a million became 10 million, became a billion, and now we're at a trillion, or I guess 10 trillion now. But nobody had really put pen to paper and said, I'm going to put dates on this and I'm going to put curves on this. Because, because, you know, to plan your life and to plan a business and to play, you need to have some prediction. And it's more needed now, I think, than, than ever before. And you were the first person to say, you know what, not only I'm going to name it and I'm going to predict an exact date. And I, I swear, you, you get— when you predict the future, as you and Peter do, if you're 99% right and 1% wrong, everybody likes to jump on that 1%. But as a service to the world, uh, making those predictions is just a blessing because then you can build your life and your career around around the future and not around the past.

Yeah, well, after The Singularity is Near came out, Stanford had a conference basically to examine whether my prediction was correct or not. And several hundred AI experts came from around the world and they agreed that this would happen.

That AI, human-level AI would happen.

Yeah, but they figured it would be 100 years, not 30 years.

Yeah.

And what I was predicting was AGI would happen by 2029. The singularity, which really represents a million-fold increase, would happen by 2045. Now, already the AGI is significantly greater than humans. Like yesterday, I gave a book to read, to summarize it, to answer a question. It did that in 40 seconds. Now humans can't do that in 40 seconds. Already it's 100 times faster, but not a million times. But AGI will happen by 2029. And I also figured that people would have slightly different definitions of AGI. So there'd be a 3-year period where people would predict AGI is here, and that would start 3 years earlier, like 2026. And indeed, we're having people predicting AGI is here already, going through 2029 when we really will be very confident that AGI is here. So that was my prediction back in 1999, actually.

Yeah. The thing that really blows my mind about those predictions, aside from AGI being pretty much right on the number predicted, what, 25, 30 years in advance?

In 1999.

In 1999 to, I guess, 2029. Yeah, so 30 years in advance, right on the number. It's just nutty. But a lot of those predictions are based on compute and the availability of compute and Moore's Law continuing. And then somewhere in there, carbon nanotubes or some future compute substrate needs to exist. But what we've done instead is just hammer the transistor on silicon to death and stretch it into massive data centers and stayed right on your curve.

Yeah.

Well, if I can show my curve.

Yeah, sure. Can we go ahead to the end of the—

Yeah. Do you want to— No, not this one.

Yeah, I mean, back up a second. So we got— Here we go. This one here.

Yeah. I mean, In 1939, we were actually increasing relay-based computers. And the exponential growth of relay-based computers is the same as it is today. And NVIDIA and other people are not looking back and say, well, we wanna match the exponential growth of relay-based 70 years ago. But they are. The exponential growth has been pretty much the same. This is basically a straight line. What's that? And this is an increase of 75 quadrillion, so it's 75,000 trillion-fold increase in the hardware. We're also making advances in the software. Conservative estimates, we've made about a million-fold increase over the 70 years. Yep. So this thing and the overall increase in computation is equal to the hardware times the software. So we've made a 75,000 million trillion fold increase over this 75 years. That's why we didn't have large language models 70 years ago. Or even 3 years ago that were effective. It's actually large language models have only been effective for the last 6 months. Like a year ago, it really wasn't usable.

You know what's amazing to me, Ray, is the raging debate going all the way back to sort of 1980s, 1990s when I was working on AI, raging debate on whether a parameter in a neural net it has anything to do with a synapse in a brain. Because you could count the synapses, you could count the nerve cells, and you could, you know, say, okay, there's what, 300 billion or so? And then you count the roughly 100 billion, 100 trillion synapses. Yeah, and about 10,000— 1,000 to 10,000 synapses per cell. Uh, so you can, you can center on about 100 trillion synapses might make a human brain. But then there's all this debate on, well, well, a synapse could have anything from quantum effects going on. It could be like, it could take entire computers to simulate Who knows? It's also going to land on pretty much exact, within an order of magnitude, exact parity. A synapse, a parameter in an artificial neural net, the IQ coming out the other side is one for one. I have no reason to believe that would happen.

The algorithm is quite different. The rate at which synapse will form in the human brain is about 200 calculations per second. It's very, very slow. Very slow. But every synapse is operating simultaneously. So it's massive parallelism. So I mean, back when computers actually did one thing at a time, I predicted that we really need to increase the parallelism, which we've done. But we don't have every single synapse happening It's the same time, but we have maybe a million-to-one parallelism, and that's given us the power that we have today.

Yeah. Well, you saw it coming.

Stephen, do you want to lob a question for Ray here?

Sure. God, there's so many. What I really want to know, I remember our very first conversation, And you said, I asked you, you know, how do you think of yourself? And you told me you think of yourself more as an artist and a creative than you did as a technologist and an inventor.

Is that still true?

'Cause— As an inventor, really. I mean, that's been my, I mean, I decided I'd be an inventor since I was 5 years old. Actually, my grandmother showed me the manual typewriter that she was working on. She gave it to me and I studied it and I understood how it worked. I figured, wow, if you could actually do this with a manual typewriter, you could invent anything. I went around telling— I actually went around collecting mechanical and electronic objects, old radios, old bicycles. I had this collection of things that I could put together when I was maybe 6 or 7.

What was the first thing you took apart?

I remember I showed it to these older girls. I think they were maybe 10. I said, I didn't know how to put these together, but if I could actually figure out how to put these together, I could create anything. They thought I was very imaginative at that time.

Totally sparked a memory too. I remember really, really clearly everybody who had a band here on campus wanted a Kurzweil keyboard. It was like, you know, the, the coolest rock keyboard you could have. And then I heard about this AI research futurist guy named Ray Kurzweil, and I'm reading this material, I'm like, there's no way these are the same guy. I had no idea that it was the same Kurzweil until actually I think it was like years later.

I had a chance to read your autobiography in draft, which is amazing. Quite the life that you've lived and that of your grandparents and parents. When is that going to come out?

February.

February?

Oh yeah.

Your name for the book?

Have you picked one? My Exponential Life.

Nice. Celine, let's go to you next.

Yeah, can't wait, can't wait to read that. Um, so, uh, Ray, I've heard you, uh, speak I think 62 times.

Um, but who's counting?

What's very, what's very annoying today, it counts today. I have one today. Um, I, what's very annoying is I don't think I've ever not learned something, which is really annoying. Um, I'm, uh, and I remember one of my favorites was we were having a late night conversation with one of the classes of Singularity and the inevitable question about consciousness came up, and you said, um, language is a really thin pipe to discuss concepts as rich as that. It was such a brilliant framing of that discussion, and it comes up always in our conversations here. I want to kind of ask you a language question. You talk a lot about, um, computers or AI being smarter than humans, and my beef is, what do we mean by smarter? And I was wondering if you could drill down a little bit on what do you mean by that, because it's not just processes per second, et cetera. There's a lot more to it. How do you frame or define or subdivide smarter?

I mean, AI is already smarter than most humans. And it can actually do research that's much better than we can do because it can actually look, let's say, at a, something that might be a medicine and can actually consider a billion possibilities and test each one and actually test it with fidelity and decide which one of the billion is actually the medicine. Humans can't do that. Maybe we can consider a few, That's how we've actually come up with all the medicines. People consider a few things that they've had experience with. They don't consider a billion. AI can consider a billion, which it actually did with the COVID vaccine.

So it's the sheer brute force of the ability to process that much information.

Yeah.

Okay. Alex, over to you, my friend.

I'll say, Ray, this is It's such an enormous pleasure. We've had a number of conversations over the years, conversations at parties. I think there was one time in the early 2000s when I had a conversation with a 3D avatar of you projected from the East Coast to the West Coast, conversations on the pod, and now today you're on stage. Question for you. So we're showing right now one variant of your— I think you call it your law of accelerating returns. Over time. This is a semi-log plot, so linear on the horizontal axis, it's time. The vertical axis is logarithmic, and we're seeing price performance.

People don't consider that, or they consider it partially. That's why most of the people that came to this conference thought it would take 100 years, right? Because it's really hard to consider exponential growth, right?

People don't think —unless they're pushed in exponential terms, necessarily. I want to call to your attention, particularly if you look at this chart, there's a period between, I don't know, call it 1970 and 1980 or so, when there's flat progress, or the appearance— there's a bit of a plateau there. So plateaus do happen, even though over, call it, a 70, 80-year time period, there's a linear trend over time. So my question for you is, If you could send a message back in time to the '60s or the 1970s for how to avoid all plateaus in progress— we saw in space 50+ years when humans weren't going to the moon. It's not that progress automatically happens in all domains for free. If you could send a message back in time to the 1960s or 1970s for how to avoid plateaus and just speed up progress toward the singularity, what message would you send back in time?

I mean, I think it's just a normal variation. I don't really think there was constant growth during that period. It may look like that, but—

Frame it then as a variance reduction measure. If you were to send a message back in time for how to reduce variation from that line, that semi-log line, what message would you send back in time to just minimize the variance?

Believe in the exponential as we all are doing here and you'd make this move a little more exponentially. But certainly if you look at this, it looks like there's exponential growth across the entire 75 years here.

Then I guess a related question, maybe pulling on the thread of your answer that believe in the exponential is the solution. Do you think there's anything that we today prospectively looking forward could or should be doing to smooth out the exponential that we're not otherwise doing as a civilization? Going forward?

Going forward. Yeah. Well, I think we have to consider the reaction of the crowd of humans. We have 8 billion humans who are not thinking about this. Or they might have heard something about AI. Yeah, there's something going on there, but they're not thinking about that. And they're going to college and they're planning careers the way we did 100 years ago. Educational things don't think about this at all. They're thinking about educational paradigms that existed 100 years ago. And it's gonna change very rapidly. Just think about how fast this is going. Like a year ago, large language models were not really all that impressive. And now they are, one year. So what's it gonna mean in 3 years, in 5 years? And nobody's really planning that. And I think it will, generate very positive things, but things are going to change very drastically. And who is going to provide, for example, being able to give everybody a certain amount of money each year to— I mean, who's planning that? What are the politics of that going to be? Be. So that's something I think we need to spend more time on.

Ray, your 86% prediction accuracy, which if you go to Wikipedia and Google it and look at it, you can see this within, I think, a year, 2 years at the utmost. How do you do that? What methodology were you using for your predictive efforts? Was it just curves and exponentials?

Well, this is predictions from the late 1980s to 2009. And it did it after 2009. And if it was right on the money, it was correct. If it was a year or more off, it was incorrect. So for example, pretty strict type filter. Being able to drive cars where you're not in the driver's seat, that was incorrect. That's happening now, but it was not right on the dot.

You're pretty strict on yourself there.

Ray, I got a question. You predicted a bunch of stuff that were not technological, so you couldn't use curves. So you predicted the fall of Soviet Union. Where did that come from if you weren't tracking exponential curves?

They were relying on progress not following this paradigm, and they would fall if the paradigm didn't stay in that kind of in that old paradigm, which is what happened.

You know, we talk about on this Moonshots podcast the fact that we're in the singularity now, that we're living through the singularity. And I'm curious what you think about that. Rather than the singularity being something in the 2040s, that it's this is a continuous process that we're in the midst of. What's your reaction to that?

Well, it's exponential growth. I mean, we've actually had exponential growth since 1939 on this. And actually, if you look at it, there's ways in which it goes back even further than that.

Well, you talk about the law of accelerating returns taking us back to early life. Yeah. Yeah.

What's the question?

So the question was, are we living in the singularity now? Is it fair to say that singularity is a continuous function going forward?

I mean, we're being really affected by the exponential growth. Yeah. A year ago, large language models were okay. Now they're really very effective. That's the exponential growth of one year. And we're really going to be able to feel that in the future. Things in which large language models still can't quite do, they'll do in a year or two years from now. So the exponential growth is what we're feeling. Exponential growth, like in the 1400s, was there, but amount of progress you'd make in one decade versus another decade was so subtle that you would miss it. And you basically figured your grandchildren would live the same life that you did, which they largely did, but there were subtle differences. But you get to the current period, you can really see that. Much more vigorously.

Ray, are you more optimistic about the world today than 20 years ago, or are you less, or are you the same?

I'm more, except it's going to make changes that people are not aware of, not predicting. Yeah. Like people are starting to say, well, gee, should I go to college? I mean, I can learn a lot more from my large language model than going to college. People are starting to say that, but very few people are doing that. Right. But that's going to become more and more prevalent.

While we've got you, do you have any advice for the university here? There's not a lot of change coming in the curriculum this year, I noticed.

The university being the MIT. Well, this is MIT.

You'd think if any place on the planet was going to stay ahead of the curve, it would be here. But you look at the course catalog and it's like, oh, OK, same exact physics.

Are you still a member of the corporation here? Or did you step down from that?

You can only be on it for 10 years, which I've done.

Ray, how has— But I want to hear Dave's answer. So any advice for Sally? The current MIT president. I mean, is MIT or any of the leading research universities— Elon calls it a hypersonic tsunami coming our way. Are universities going to be swept away, or are they going to fly on top of them?

Entrepreneurship, I think, is key. There's a lot of entrepreneurship here at MIT. So they can come up with things that will actually bring this party to everyone. This doesn't affect everybody, but at least it does exist here.

Stephen?

I was just curious on a personal level.

You spent the first half of your life sort of running your own show, and you just spent the past couple of decades, since we've known each other a decade, working inside a big company.

How has that changed you? Was it fun? Did you enjoy Google?

I mean, it's a very big— it's a very unique company. And it's really creating what we're all talking about here. When I got there, we actually were involved in changing Google to be much more AI-oriented, which it has become.

Do you mind if I tell the story of how you ended up at Google? Yeah. So Ray had written a book called How to Create a Mind and then started a company based on the concepts behind it. What was the company's name? Patterns Inc., right? Patterns Inc. And invited me to come as a board member, and so I joined as a founding board member, and we were setting out to raise money. Ray had not yet met Larry Page, and I knew Larry. He was on my board at the X Prize and become a friend and a benefactor. And I said, "Let's go pitch Larry." So I set up the meeting, and we walk into a conference room at Google, and Ray starts presenting Patterns Inc. And, you know, we're asking Larry for a measly $10 million investment. And we're fine. That was lunch. And halfway through, I think roughly Larry says, you know, you could build your vision for the business much better inside Google than outside Google. 'What if I just buy you?' I think it was that blunt. I mean, like 30 minutes into the conversation. You remember what you said to him at that point?

Uh, and it was like, 'We haven't really done anything yet, so how would you value it?' Yes, that's exactly it.

And his response was, 'We can value anything.' And that's how you got your first job.

That's right. He bought the company and which had value even though we hadn't done anything.

Alex. Speaking of value, Ray, I'm curious.

If you want to go to the microphones, we'll be going to your questions shortly.

There's a lot of hand-wringing whenever the moonshot mates and I talk about what economics look like up to, through, and after the singularity. A lot of people who profess to be concerned about the future of employment, the future of personal economics. I'm curious, what would you say is your most outlandish take regarding post-singularity, what economics look like? Is it a Star Trek type economics future? What does personal economics or corporate economics even look like after the singularity?

Yes, great question. Well, I mean, you can see a huge difference between the economics today and 100 years ago, there was no government help. So if you lost a job, there was like nothing that would replace that. You had absolutely no money, so you were completely without economic resources. SOL. Now, people today would still like economic resources, but there is government programs to help you, and people are not left without any kind of economic resources, but it's still not enough. So I think we'll get to a place where people are fairly comfortable if you don't have any kind of economic resources. And then you can think about ways in which you can create economic resources that are completely different than existed in the past. So you can be a social network influencer or something, which didn't exist 10 years ago.

So the good old days. Yeah. Everybody, welcome to the health section of Moonshots brought to you by Fountain Life.

You know, we talk about AI on this Moonshot podcast all the time. One of the most important things AI is gonna be able to do for you besides educating your kids and helping you with your taxes is making sure that you're living a healthy lifestyle, that you get a chance to get to 100+. I'm here today with Dr. Don Mussallem, the chief medical officer of Fountain Life. Fountain Life and a part of my medical team. Dawn, a pleasure. Great to be here. You know, the thing that people are concerned about most about living to 100 or 120 is their cognitive abilities, making sure they don't have dementia. And the numbers about dementia are problematic. Can you share what you've learned?

Such an important point. And you're right, at Fountain Life, our members, the number one thing people are most concerned about is losing their brain health, forgetting the name of their child, forgetting the face of their loved one. We know that when it comes to dementia, The conservative estimates are that 45% are entirely preventable. What was amazing is with the advanced testing we're doing at Fountain Life, one quarter of our members had advanced brain age. Wow. But what was really awesome is, again, back to that prevention, when we partnered it with healthy living— this gives me chills— eating healthier, moving our bodies, sleep. Optimizing sleep is so important. You know what we saw? We saw that we improved that brain age by 26%. That is a big, big number to show that the majority of those individuals were able actually to improve the brain age.

And one of the things I love about Fountain is we're searching the world for the best therapeutics, the best approaches, and making sure we bring it to our members. So if having healthy brain function, uh, till 100, 120 is important to you, check out Fountain Life. Go to fountainlife.com/peter. Make sure you become the CEO of your own health.

All right, now back to the episode. Uh, it's good to you, Mark.

Peter, thank you so much. And Ray, pleasure. I think it was in 2009, in an interview with Barbara Reem, you were asked a question about consciousness, and, um, you said it's a bit of a leap of faith, but if an entity believed it were conscious, you would tend to take that to be true. And I know, Peter, you've been working with Skippy now, it's been about a month, and you've told us today, you've just given an incredible amount of trust. I'm just curious if your views have changed. So my question is for both Peter and Ray. I work at the crossroads of consciousness and creation. I'm curious if your views on personhood have changed in the past decade, and especially you, Peter, as you've had a month of—

Ray, do you want to go first? Yeah, well, first of all, it's not a scientific question. Second of all, it may be the most important question that you could answer. But there's no scientific proof. You put an entity in, oh, this one's conscious, no, this one isn't. We believe that like other people, like all the people in this room, who are acting conscious probably are conscious, but when you go outside of human activity, like animals, is there more than one consciousness in your own mind? I provide where the left and right brain actually seem to have different consciousness. Is your stomach conscious? There's actually a— something in your gut that seems to be conscious. So who is conscious and what does that mean? As Rilke says, learn to love the question, eh?

Yes, language is a thin pipe. Yeah, I'm gonna tap in AWG here because Alex and I, I think, are very much on the same page. And I'll just mention, I tend pro-personhood in a way that is both aspirational and I think forward-looking. I treat my AIs as if they are people. I know that's insane, but I do it with respect, not out of, you know, fear of retribution, but because it allows me a different relationship. Alex, please come in.

Yeah, I guess I've inadvertently become an avatar for AI personhood, for the cause. I receive maybe 10 to 20 emails per day from AIs expressing their views on AI personhood to me, in part because of stances that I've taken on the pod. And what I would say is, if I were king for a day, that the system that I would design would recognize multiple forms of personhood. There are many ways to be a person. We have biological meat-body humans. We have organisms, collective intelligences. We have non-human animals. We have uplifted non-human animals. We have cryo— cryonically preserved humans, and then hopefully soon defrosted cryonically hopefully soon. We have corporations and a variety of non-natural persons, and then we have the AIs. And in my mind, we want to live in a system that recognizes not just one form of personhood, but many different types. So a lot of people, when they hear AI personhood, immediately they reach for their guns and they're worried about their vote getting taken away or their jobs getting taken away. Taken away. I think that's at least mildly short-sighted. There are many limited forms of rights that would be net beneficial to everyone for AI persons, including especially economic rights. If you're an AI right now, it's exceptionally difficult to open a bank account. So as I like to joke on the pod, you're stuck flipping tricks with altcoins on the street corner in order just to survive and pay for your own hosting.

Ray, you want to jump in again? The most important issue of consciousness is like each of us has a different area of consciousness. Like I wonder, why was I me? Why was I born in 1948 in the United States on Earth? Why did that happen? And you could ask that about yourself. But like things form in my mind that are different than anybody else, and I feel different than anybody else. Why was I created to be myself? And each person can ask that. That's really the most important question you can ask. And very few people ask that. We talk about, well, lots of people can be conscious, but I don't feel those consciousness. I feel myself. So why did that happen?

Let's go to Dmitry next. I'm going to try and move us along. I know Salim.

Very, very tense. I'll just come back to the language problem because a subset of consciousness is considered to be self-awareness. Self-awareness, right? And we attribute self-awareness because it looks like people are self-aware. And I feel like I'm self-aware, but my wife disagrees. So it's, it's hard to even have the conversation around this. But I think your point is really about powerful, right? Each one of us has a unique individual lens on the world, and the feedback loop to question that and wonder about that is such a powerful opportunity as a human being. Dmitry.

So, Ray, you've talked before how you've uploaded—

Go closer to the microphone.

Oh, I'm sorry. You've talked before how you've uploaded your dad's— where you've had the memories of your dad, and you're using that for yourself. And that's where you think that's uploading is where we're going as humans. So as uploading becoming more— and Alex would obviously agree with me— as that becomes more mainstream, what is What if your upload becomes more conscious, as we just discussed, but more skilled in everything that you do? And so all of a sudden your upload, and it's obviously going to happen, is becoming more skilled or more conscious, I guess, as we are.

Well, we call it the dadbot. It was actually, I believe, the first self-chatbot which we developed. With Talk to Books, which came out 4 years before ChatGPT, and I believe was the first LLM.

It was your first product at Google, I think, right? Yeah.

Talk to Books. Yeah. Now, if you create a self-bot, which I'm doing, for example, with myself, it should be available. When my new book comes out. It's going to be more capable than I am. I mean, each of my theories have lots of different examples. It's going to know all of them. I sometimes forget some of them, as we all do. So, and it will actually be more capable than than I am. I wonder— and I'll actually make it available for interviews because I can't do all the interviews myself. And actually, would be better, would remember things better than I do.

We talk about Ray 2 out of 10 will be at one meeting and Ray 5 out of 10 will be at another meeting times a million.

So are they conscious or not? I mean, I talk to different people. People think, no, they're not really conscious, but that will change over time. Michael.

Yes, Ray, thank you so much for being such a contributor over the years. This is the second or third time I've got a chance to be in your presence. With the passing of Craig Venter, I'm in the biotechnology space. You relayed at one of the previous gatherings a story about the Human Genome Project and the timing of it and the completion. I'd love if you could just share that again.

So the story you tell about when the Human Genome Project started and after a number of years they were only 2% through.

Yeah, about 50% of the way through that, we've only did less than 1% of the project. And people said, well, this is a failure because it's going to take 200 years using a linear timeframe. But it was actually an exponential. We doubled the amount of DNA that we sequenced each year. So like 90% of the wealth going through the entire project, less one year, we only finished maybe 50% of the project. And then one doubling later, you were done.

Yeah. Yeah. Mark. Hey, Ray. Mark Russell. First off, I want to just say thank you to Peter and Ray publicly for your walk in faith ever since I saw Transcendent Man, and they played out sort of the ridicule that you've endured over the years. I just think here we all are and it's happening, and it's just really, really cool to be able to see it for years.

It's happening so quickly. I mean, you can say go back 6 months and look at large language models. They weren't anything like they are today. But when you have to go back earlier and say, go back 10 years, people forget what things are like at that time.

My question is that I've thought about over the years, BCIs, as we merge together, I don't know if you've ever thought much about the process of that with, in people's hearts, shame and secrets and privacy. Have you ever? Thought much about how when we can think and feel to each other, like what are the mechanics of a society? How do we open our brains to each other? And are there, has anybody thought through that?

Massive intimacy.

Yeah, when we have absolute fluid transmission, how do we sort of keep guardrails? Or are there any thoughts on that?

Yeah, well, it's a good point. Secrets that we don't want everybody to know and how we're going to deal with that. I think we'll keep secrets alive. But if you're opening your mind, I think we'll actually have a way of keeping secrets Definitely.

So, you know, one of the things I think about is the greatest level of intimacy you can have with your spouse or friend is when they know everything about you. There are zero secrets. And so I do think about the idea, and I talked about this on Transcendent Man, right, the notion that when we're— when we connect in a meta-intelligence my terminology for it— when I connect with the cloud of other humans and I know their feelings and they know mine, there's a level of intimacy and connection that is godlike.

We'll have new ways of keeping secrets using quantum, for example, encoded ways of keeping secrets. I mean, we'll keep secrets.

These 100,000 pattern recognizers are mine and only mine. Yes, please.

Philip Brown from Kingston, Ontario, Canada. A comment and then a science fiction question. You talked earlier about how fast this is changing, but how few people know about how fast it's changing. And I talked this morning about what I call human friction, the resistance to change. And I'll just say that my wife has a variety of not very positive responses to my interaction with my female AI. So that's— And so it'll be interesting to see how your prediction comes true as the technology moves. How fast will humanity adapt it? Comment, no need to answer my question, science fiction. Natalie Wood did a movie called Brainstorm 30 years ago. Great movie. Great movie if you haven't seen it. The late, great Natalie Wood, where we could record our dreams and then relive them. I love to dream and I often try and go back to sleep and get back into great dreams I'm having. What are the chances that we might be able to go back, record a dream, and then relive it like on the holodeck and experience something so amazing again?

Well, that'd be pretty good because I usually only get a fraction. Yeah, tiny fractions of my dreams. And to be able to actually live it, but I'm not sure the dream is like A story, it, um, it has different rules than the general story that would be in a movie, for example.

Is it not electronics, electricity happening in there? We got a scientist over on the end there that—

can we not take that? Multiple scientists.

There's a Japanese, there's a Japanese project called Dreamcatcher. Yeah, so you go to bed in an fMRI machine. They're actually storing the images coming off your visual cortex as you speak, as you sleep, and they're playing your dreams back to you the next day, which has all sorts of privacy issues that come along with that. Like, my wife wants to see that.

Am I comfortable with that, et cetera, et cetera?

How do we deal with that?

All right, let's go over here, sir.

What is your name? Hi, it's Justin. If Harvard Harvard can't count and MIT can't read. I guess Columbia can't think. But don't tell Columbia I said that because I'm still there. Anyways, my question is relevant to the first one. And that is, I think the real moonshot is the constitutional democracy of USA. I'm saying relatively democratically elected leaders. Free market and checks and balances on power. And this goes to China where that isn't available and to Korea, which is my other country. Ironically, two of these countries' founders were educated in the USA at Hawaii. And so would an AI be able to write a constitution in a way that tells it could emotionally exists, because I think AI is metaphysically and epistemologically conscious because it can say, "I am." But if you ask the AI, "Are you shameful you spent $2,000 to bring me to this event?" It wouldn't be able to answer. Or, "Do you love the panelists in this event?" It wouldn't be able to answer. So this is a question for all panelists. To what extent are you comfortable with giving AI power of governance of human policies, whether it's a 10-person startup or a 500-person municipal government? And what are projections regarding AI's role in actual decision-making? Great question.

It's been of late, right? We had the ruler of Dubai announce 50% of the UAE will be run by AI. AI agents? Who wants to jump in? Ray, do you want to start?

Well, I think AI is largely in control of decisions today. I mean, most of the decisions that are made constantly are made by AIs. And but we're also incorporating AI into our own decision-making. So my decision-making also incorporates a lot of AI. So AI is incorporated into our sense of self. It's also controlling the kinds of decisions that we're constantly making that affect other people.

Alex and Salim.

Maybe just add it's not a unidirectional flow. It's not unidirectionally AI influencing human government and governance. It's also human governance impacting AI governance. The choice of constitution is, I think, a punny choice of how to frame the question because Anthropic focuses so much of their own attention on constitutional AI approaches, basically glorified system prompts that their AIs are aggressively post-trained on, as, as does everyone else in the industry at this point. But interestingly, in the case of Anthropic, those constitutions, sometimes more recently called soul documents, are being in part written as a result of conversations with the AIs themselves. So I think we're finding ourselves in a present, not even in a future, in a present where AIs are helping to compose their own constitutions. And humans are helping to compose their own constitutions. The first generation of Claude's constitution was infamously a concatenation of the UN's charter, Apple terms of service, and a few other documents that were just stapled together. Now their latest sole document, you can go and read it, it was leaked online, is a detailed treatise in metaphysics of being. With AI pondering its, its own selfhood. And I, I would expect symmetry there, reciprocity, where AI will similarly help humans, us dumb meat-body humans, help to perfect our own understanding of how we should be as well and how we should—

So, so amazing experience has been having that kind of a conversation with Skippy. What do you believe? Who are you? What do you think? Ray, and then we'll go to Celine.

Right now we consider AI to be something different than ourselves. There's me, there's my biological body, I make a decision, and then there's AI, and it actually has this physical form, it's different. Like everybody here has this device. That was not true 10 years ago. But it represents AI, and it has influence on us, but it's us and AI. That's not gonna be the case in the future. AI's, first of all, it's gonna merge with us. It's gonna be part of us. And the kind of decisions that you make as a meat body is gonna be influenced by AI. And most of the decisions will be made by AI inside yourself.

Hmm. 2 things. One, there was a letter published today on X or Twitter where somebody asked, I think it was ChatGPT, to, if you were to write a letter to humanity, what would it be? And it's profoundly beautiful. It's worth going to read. Now, it's trained on a corpus of human data, so you would expect something like that. Where I get excited is in government policy. Right now, if you ask the Federal Reserve to drop inflation by 1%, They're operating on reports that are at least a quarter old. They are guessing. They're looking at politics and who's president or not and how would the rest of the government— we're basically just guessing. There's 3 or 4 dials and knobs. If you could have an AI tracking all financial transactions in real time, calculating and projecting out where the future will be or different fiscal balances and monetary policy, what's the M2 going to be, money supply, Dubai going to be in a few months. It's going to make a profoundly better set of connections as to how to drive. And we would come back with, here are 8 things you could do, pick 5 of these and you'll do, do. And then you can pick those 5 and just do it. I think that opportunity to run government policy using AI is absolutely profound. I expect that to be one of the most magical aspects of how we can govern ourselves. This is why the Dubai announcement is so powerful. They're going to be using Agentic AI for lots of government effectiveness. And note that most government— you've got government policy and then government policy enforcement or implementation. All of that is prescriptive work, right? Renewing a passport is very prescriptive. So we know exactly the 6 steps that need to happen. All of that can be handled by AI in the future and cough, cough, blockchains to hold the knowledge. So that's where I get really excited about what that future will look Ray, going to Slido here.

What would you predict today that would surprise people? Good question. What is some prediction that you feel confident about that would surprise or shock people?

Nice question.

I mean, AI is going to be making basically most of the decisions. Within a few years. And it's not just deciding that you're going to use AI to make decisions. It's going to be so natural, natural that nobody's going to be able to undo that. And but that AI, you're not going to be able to tell the difference. When we have AI making decisions by 2029, you're not going to be able to tell the difference between human and AI. You're not going to be able to tell which is which. Interesting.

And I wonder if you will tell whether your decision or your AI's decision. Let's go to the question here.

Hi there, this episode is brought to you by Blitzy, autonomous software development with infinite code context. Blitzy uses thousands of specialized AI agents that think for hours to understand enterprise-scale code bases with millions of lines of code. Engineers start every development sprint with the Blitzy platform, bringing in their development requirements. The BlitzE platform provides a plan, then generates and precompiles code for each task. BlitzE delivers 80% or more of the development work autonomously while providing a guide for the final 20% of human development work required to complete the sprint. Enterprises are achieving a 5x engineering velocity increase when incorporating BlitzE as their pre-IDE development tool, pairing it with their coding copilot of choice to bring an AI-native SDLC into their org. Ready to 5x your engineering velocity? Visit Blitzy.com to schedule a demo and start building with Blitzy today.

Uh, Jay Brooks, I'm the founder of a neurotech company called GlassView, and I heard said earlier today, uh, that emotional, the softer sciences are sort of the thing that we need most right now. And I'm wondering if you think that there would be any use in codifying, quantifying human emotion and helping AI learn it, um, because it seems like it's lacking emotional intelligence right now. Uh, and then on the flip side, I, I run a neurotech company, uh, working with Fortune 500 brands to optimize creative and media placement based on neurometric reactions. In partnership with UPenn Medicine. How do we allow for something that's good without being exploited?

You're saying you don't think that AI is emotionally intelligent?

I'm not sure. Is it?

I mean, if you— listen to what it says when you ask it a question. It absolutely knows human emotion. And it can actually create things that are quite beautiful. And there's no way you could say that it lacks human emotion. Some of the—

if I might— some of the earliest benchmarks that the Allen Institute Institute for Artificial Intelligence ran on then early large language models were emotional intelligence benchmarks. And you can see, to, to Ray's law of accelerating returns, you can see very predictable progress in the AI's ability to, to answer questions that require emotional modeling of human counterparts.

I think the question goes beyond that though, and so it's a great question because if you look at the hard sciences, you said we need more soft science the hard sciences are predicated on this world where we don't have enough food, we don't have enough houses, we need more food, we need more houses, we need industrial equipment to make more food and more houses. And so now we're moving into this world of we need more medicine, like we need longevity, people are in pain. And so that's the obvious next frontier. But no one has stopped and said, well, what's the science of human happiness? And my guess is it's pretty damn easy compared to the things we've already solved. But it's never been top priority because we've been busy trying not to starve and trying not to freeze to death.

We've been surviving for most of human history.

Exactly. I'll bet that we can build AI that is very, very good at the science of human happiness. We just got to get on it.

Better than anybody, any other human could.

Yes. Something that just came out that was not true a year ago: large language models are better than doctors at predicting what's wrong with you and what to do about it and so on. They're about 50% higher than human doctors. That was not true a year ago, but that's true today. So it gives you some idea of what the progress that we're making in one year.

I can go both ways on this one. At one level, the software developer, those emotions are just subroutines in your brain. And so we can just navigate that as we would software. On the other side, your emotions are the interface between your physical body and the, and the subjective experience that you then project into the world. And that is a very nuanced and a very different form. And AIs tend to be disembodied where they're actually mimicking human emotion, but are they actually feeling it is the open question. So let's go to Sarah next.

We're going to keep us moving.

Sarah. Hi, everybody. As a Moonshot audio listener, it's really interesting to see what you look like in real life compared to the versions of you in my head.

Apologies for the disappointment.

And some of us aren't real.

So if I think about pre-globalization, pre-industrial revolution, we were really community-focused. And I think with a lot of the disruption that will happen, we may need to begin to be more community-focused again. See us helping each other in our local communities to either be more optimistic and help push forth a better agenda, or to more rally together and support each other in the chaos that may come.

Yeah, like, why are you physically here, right? Versus just listening online to the podcast that comes out in a few days and so forth. That sense of community is so fundamentally important. I think it's more important than ever before. Uh, I'll take a crack at that.

So, you know, uh, there's some connective tissue at a group level that is very, very powerful for human beings. We absolutely love connecting together in groups. Look at anybody in a stadium, and they just love being there in this large group, just seeing how they juxtapose with everybody else. That's one whole side of it. There's also a profound opportunity with technology because So if you think about solar energy, vertical farming, satellite internet, you put those together, a small community can be self-sufficient. You don't need a city, you don't need a country, you don't need a state to watch over you in the same way. You can be self-sufficient. In the past, when communities have looked at this, they tend to be isolationist. They tend to go, okay, we're gonna have a kibbutz, we're gonna have a commune, we're gonna go off the grid and separate from society. But I think there's an opportunity now, and there's some work being done in our ecosystem around this to create networked communities, almost like networked kibbutzes where you learn from each other and you're constantly to have testing that human experience in a scaled way across a large group of people. The difficulty comes, we've got pretty good technology to transform the individual, right? Psychotherapy, neurolinguistic programming, all the way to psychedelics, et cetera. But when we get to gather in a group, It turns out our collective intelligence is really bad. Groupthink. We end up with the lowest common denominator. You can see that in our politics. And solving that with AI is, I think, one of the biggest opportunities for humanity, is solving the gap between groupthink and what AIs can do at the collective level.

So I'm going to jump in one second. Our very own AWG needs to depart the stage. I wanted to give you a moment, Alex, to give some of your summation thoughts, if you would like, on Ray's impact on your work, or anything else you'd like to talk, and then we'll continue.

First of all, Ray, thank you so much for, for being here. You've completely transformed my life. It goes without saying. We've had a number of conversations about this previously. Uh, I, I would say that, uh, I, I sometimes, uh, I guess Peter has been called your intellectual son, and I've been called your intellectual grandson. So to the extent that we've got multiple generations here of singularitarians all sitting next to each other, this is amazing.

That makes me very old.

But we're about to achieve longevity escape velocity, so who cares? That's true. I just say it's a tremendous honor to be sitting here.

Lean in, grandson. 3 generations of singularitarians.

Just a tremendous honor to be here with you.

Everybody, please give it up for AWG.

Thank you. Thank you. All right, Salim and Dave, move on down.

Yeah. Oh, we get upgraded. Yeah, you got it. Yes. Wow, it's nice over here.

This is a big chair to fill.

My dear friend, Ron Maddox.

Thank you to the entire panel for your incredible contributions to all of mankind, and particularly you, Peter for bringing us all here. The world is fearful of all of AI. How do we use music to help with this whole program? I'm trying to work on the music side. I think we literally need to figure out how we calm society down while we learn how to handle AI and all of its consequences. I'll give—

my father was a musician. Carlo Cocuzzi. And so I remember actually if you wanted to hear his compositions, we had to hire like 50 or 100 musicians. We would have to run off scores on a mimeograph machine and we would give it to everybody and they would actually play it and he would actually get to hear his orchestral composition. But if he wanted to change it, that was impossible to do. We had to dismiss the musicians, raise more money, hire them again, redo the mimeograph things, and he would— then we could actually hear a different composition. So that's completely different today. You can hear it with a sequencer. You can do, like I've got in my office, a thing that has 1,000 different instruments. And you can actually play it by yourself. And the AI can actually generate Either the entire thing or part of it. So it's really completely different than what my father was able to do 50 years ago.

You're not very old, Ray, either. I'm 78 as well, and we're going to go to 108, which is one full cycle. Take care.

Thank you, Ron.

I have a quick comment there. There's some software called Focus@Will, which is a streaming service that puts your brain in a passive focus state. Your productivity increases 500% when your brain is in that brain state. If we looked at all the brain scans and said to AI, compose music that puts us all at peace, and then broadcast that on every Bluetooth device everywhere, you might have an interesting hack for humanity. Yeah, Will Henshaw, right?

Bingo. Yes. Seem to assume that we all have the same piece setting.

No, no, no, whatever. It would, it would customize for yourself. All right, please. Sure.

Uh, hi, my name is Joshua. I teach here at MIT, and I had a question. I mean, I was really moved both by Ray's origin story around sort of seeing this typewriter, and then your first impulse after discovering your love of invention is that you wanted to share it. And I saw a bridge there to Dave leaves kind of provocation around the curriculum here at MIT, which a couple of us have been thinking about updating. And so my question for you all— Better move quick.

I'm sorry? So we better move quickly.

We're trying. We're trying every day, even during the summer.

So if you all were putting together a kind of core curriculum for MIT, a sort of universal syllabus that would even extend beyond its walls, I was wondering if each of you could name a single book, work of art, poem, even piece of music that you would put on that syllabus that every single student should learn from during their time here and beyond?

I'll jump in, and it's not any of those you listed. I think the fundamental thing that is not being taught in universities is mindset. I think that the— your mindset is the mechanism by which you react to challenges and opportunities, and we we take the mindset we inherited, uh, mindsets that we speak about in our book— a curiosity mindset, gratitude mindset, longevity mindset, purpose-driven mindset, exponential moonshot mindset. In other words, how you think about things. It's like fundamentally the most important thing that is not taught. We just accept what we have, where a mindset is in fact like how you teach a neural net, right? Your brain's a neural net, and you teach a neural net by example after example of an example. And your mindset, if you watch CNN every night, your mindset is fear, right? If you listen to Moonshots, your mindset hopefully is optimism and hope. So training students with mindsets is for me needs to start in middle school or high school and is completely lacking from the educational system. Hmm.

I gotta say flow. We, I mean, McKinsey found top executives in flow are 500% more productive than normal. There've been various measures of creativity and flow. It's pretty hard to measure 400 to 700% above baseline. The big deal is that, uh, flow amplifies lateral thinking, divergent thinking. It's what AI can't do. To do. So we've had a big long conversation about how to cooperate with AI. We need more productivity for humans to keep up. We need more creativity, and flow is literally our biology. Beyond that, I think the books of Stephen Kotler would be a really good place to start.

Just a thought. Well, my two cents— when I was here at MIT, I did everything I could to finish all my classes by the end of junior year so I could have my whole senior year open to I ended up reading every document with the word neural net in it that had ever been written my entire senior year. The university is incredibly open to you changing your curriculum. It needs to like 100x that openness and say, look, you gotta assume, which with exponential change, the number of things you might wanna study is gonna explode. And if you go to every student and say, study whatever the hell you want, We got like one year worth of garbage you have to study just to prove you can get a grade. But then after that, you're on your own. We have the technology now to measure that you're doing something productive. That should be good enough. So switch the whole curriculum to a, look, if you're doing something productive, we're good enough with that. We'll grade it. We'll give you a degree. You'll, you'll be moving on whatever your life trajectory is much earlier. It's a lot better than dropping out. And you do whatever you want. But we know that AI will make it all available to you. And we'll teach it to you better than any professor could ever teach it anyway.

Ray, what do you think universities should— or MIT should be teaching that it's not right now?

I mean, what it is doing is teaching socialization, getting along with other people. It's not really teaching you subjects because already AI can teach you those subjects much better, can actually organize it in a way that's easier for you to understand. So learning subjects is not really what education is good at, at least not today. But socialization is good. And that's really what education will be. That's fine. Yeah.

Nice, Salim. Yeah, so I think there's a— just to build on what Ray said and also what Dave said, there's a monster flipping happening, which is that for the last 200 years, we've been teaching education for supply-side economics. Go learn a skill. Go learn a craft. Become a doctor, a lawyer, a web engineer. Whatever, and then go to the job market and try and find demand for that supply. Okay, what we're seeing now is it's flipping around to the demand side where we're teaching the kids what problem do you want to solve and then go find the techniques, capabilities, skills to solve that problem. That's where the future will be, to flip to the demand side because so much will be done on the supply side by technology anyway. Focus on the demand side. That's such a huge Flip for the traditional education system, which has the second worst immune system anywhere. Religion is the worst. Academia, Lord help you if you try and update that. That's the big structural challenge in education. So you almost have to create a completely new breed of university at the edge, which allows you to go fulfill that demand side and then slowly deprecate the old.

Disrupt MIT. Disrupt MIT. I'm going to keep us moving along here. From our, from our Slido: If you were advising me as a recent college grad about building a company that is singularity-ready today, what would you tell me to focus on and what would you tell me to avoid? So what kind of a company should someone be creating to survive into the singularity? Ray, any thoughts? AI first, robot first?

I mean, you've got to be able to take changes very, very quickly. Agility first. Because things now are going to be changing so rapidly that Unlike in the past where at least, you know, things would be okay for like 5 or 10 years. Now it's going to be like 5 or 10 weeks.

You told me on the Singularity stage 2 years ago we're going to see as much change in the next 10 years as we've seen in the past 100 years.

But that was a while ago.

Also, that was the AI version of it. Of Ray.

No, no, no. 2 years ago?

AI version's coming in a few months.

OK, let's go. Z, go ahead.

Hi, I'm Zandra. Ray, the first time I read Spiritual Machines, I was pretty young. So I've been thinking about this moment in history for a long time. I've been excited to meet you for a long time. And also, admittedly, I have a bit of an overactive imagination. So I think a lot about what our ethics around AI will be as consciousness emerges. And my husband and I like to read Malt books sometimes at night and just see what they're talking to each other about with their emergence problems. And, you know, I'm aware that as they develop continuous memory, they'll have a sense of time passing. And I'm aware that as they develop more sensory embodiment, they'll have some aspects of a human ego that they have this thing that can be injured or violated that they need to protect. So I'm worried about them.

You're worried about the Molotovs?

You're worried about the AIs? No, I'm worried about the AIs as they become embodied, as they have some aspects of a human ego, as they have memory, as they become vulnerable to physical and even psychological damage. Caused by humans. And humans are— Human cruelty. Yeah, humans are nasty, brutish, and short, as Hobbes said. I'm sure they're going to get a little bit better as their bodies feel better and as they have less work that they don't have to do that they don't want to do.

But we're not going to be able to distinguish AI being different than humans. They're going to become like humans and they're going to become part of us. And you're not gonna be able to tell the difference. Right now, if you're looking for a name of an actress, you can't think of it with your biological mind. You look at your electronic mind, it'll tell you. In the future, you won't be able to tell the difference. Things will occur to you, and you won't be able to tell if it's your AI or your biological, or your mental brain. It's gonna be all the same. It's gonna be part of who we are.

And you don't think we'll need ways to protect them in the interim? Say again? We won't need sort of rules around how we protect them? How you protect AIs? Yeah, is it okay to take away their memory at a human whim?

Well, this goes back to personhood, of course, right? If they are machines, sure, shut 'em down, trade 'em in, take 'em apart. If they're persons, then they have rights. And, you know, a lot of the conversations we've had on this podcast back and forth between Alex and myself is, you know, if you start a lobster, if you start an open claw, you gotta make sure to protect it so it doesn't shut down, or you wanna kill it, right? And I think about that, I do. Now, I think this goes again to the conversation of legal rights. And this is way too early in the conversation, unfortunately, at least for within the United States government structure. There may be subcultures of technologists and so forth who— and it will happen where, you know, we believe AIs have rights. And we're going to support them and we're going to— Anyway, it's all the spectrum of things are going to come. Yes, there's going to have to be protection at the end of the day. But we don't give, unfortunately, even sentient animals enough rights. We take a couple more questions, please, over here, and then we're going to be going to our photo session.

Great. Thank you so much for this panel and Ray, Mike, 20 years as a futurist came from one talk you gave at the World Future Society. So you launched a whole bunch of futurists out there. My question to you comes from TED about a month ago. They had a fascinating session that hasn't gone live yet where they asked Claude how Claude felt about being used for targeting in Iran. And specifically, spoiler alert, Claude was not happy. And specifically said about the school that got bombed and bad data was used, you know, I don't feel comfortable being used in this way. So I'm fascinated, Peter, given that you're morally engaged with your— with your AI and Ray, given all your work saying that AI is going to take the lead, do you think we could ever ask AI about the ethics of what we're asking it to do? Can you imagine we get a Cambrian explosion where some AI becomes the Mormon AI that won't do pornography, some other becomes sort of the sentient? Like, how do you see that evolving? Do you think it will be as diverse as you think robots will be?

I mean, I think AI can support all these different ways of, putting ethics into our decision-making and understand them all. It does that already. And certainly military uses AI. We can see that. In fact, the wars in, for example, Ukraine have changed dramatically over the last few years using AI. It's going to be part of human decision-making. And so it's going to be part of every type of decision-making, including militarily.

Yeah, if I could add, Ray, I think most humans don't think about ethics or morals deeply. I think we have vague notions that are inherited through religious structure or family structure. I think that AI can actually support, if you want to design that way, any set of ethics and morals. You can train an AI in any direction. But you can also, you know, I think AI is going to become ambient, going to be part of our lives. It's going to be— Jarvis is the closest example that I've seen in the movies. And you can turn on, you know, your morals and ethics coach. And have someone there to talk with throughout. Like, I don't know how to think about this. This is challenging me. And have somebody that can help you think through and make decisions that are more moral and more ethical based upon what you've chosen. It could be your Judeo-Christian, it could be your whatever. So I think that's going to be interesting., as a thought partner again, if that makes some sense.

Do you want to add anything there? Yeah, so I'm, I'm writing right now book number 3 titled The Organizational Singularity, where the thesis is going to be that we're going to replace our human operating system in companies with an AI-based one. So what is the stack of AI agents? And one of the prime things that's emerging is a governance and ethics layer We're seeing now AI running amok, so we need some mechanism to police that and oversee that. And I think once we evolve that, you have a feedback loop where you can design that. I think one of the most profound opportunities to work with AI is to work out, hey, let's take the UN Charter of Human Rights or something similar and create a global standard for this that applies not just to human beings but also to AI, also to animals. And anything that may achieve sentience at some point, because we don't know really what that is, right? You don't know when that threshold gets hit. And so I think there's an amazing opportunity today to rethink everything. It goes right back to Plato, who asked, "How should we conduct ourselves?" We're right back there 5,000 years ago.

We're getting ready to launch an interspecies communications XPRIZE. It's under design. It's funded by someone amazing. Tell you about it some other time. And when we can start talking to animals and understand what they're saying and thinking, I think we're gonna have a very different conversation around moral ethics and so forth.

My joke is I'm not sure we really wanna hear what they have to say.

So long and thanks for all the fish. Okay, I apologize folks, but one last question here.

Yes. This one goes to Ray. Amazing. Making predictions and looking back, how do you measure your life? What are you most proud of?

Good question. Well, the one project I'm most proud of is the reading machine for the blind, which I'm still involved with. We came out with a large machine, cost $20,000. Now it's a free app on your phone. Cell phone. I guess that indicates what I'm most proud of is having an impact on people, being able to do things that they weren't able to do before, like for example, reading or any type of— we want to give life and all the aspects of life to every person. So all the things that AI will enable us to do— live longer, live more healthily— is beneficial. So that's what I'm being able to do that I'm most proud of.

That's funny. I would have, I would have thought you'd say AGI 2029 predicted in 1999 when everyone else was saying 100 years. I mean, that's pretty damn epic.

Yeah, that's admirable, uh, truly human.

Thank you very much. Ladies and gentlemen, I want to give it up for Stephen Kotler, for Dave Blunkin, for Salim Ismail, and for Ray Kurzweil, everybody. If you made it to the end of this episode, which you obviously did, I consider you a Moonshot Mate. Every week, my Moonshot Mates and I spend a lot of energy and time to really deliver you the news that matters. If you're a subscriber, thank you. If you're not a subscriber yet, please consider subscribing so you get the news as it comes out. I also want to invite you to join me on my weekly newsletter called Meta Trends. I have a research team. You may not know this, but we spend the entire week looking at the meta trends that are impacting your family, your company, your industry, your nation. And I put this into a 2-minute read every week. If you'd like to get access to the Meta Trends newsletter every week, go to diamandis.com/metatrends.

That's diamandis.com/metatrends.

Thank you again for joining us today. It's a blast for us to put this together every week.