Nick Bostrom: Simulation and Superintelligence | Lex Fridman Podcast #83

The following is a conversation with Nick Bostrom, a philosopher at University of Oxford and the Director of the Future of Humanity Institute. He has worked on fascinating and important ideas in existential risk simulation hypothesis, human enhancement ethics, and the risks of superintelligent AI systems, including in his book Superintelligence. I can see talking to Nick multiple times in this podcast many hours each time because he has done some incredible work in artificial intelligence in technology, space science, and really philosophy in general. But we have to start somewhere. This conversation was recorded before the outbreak of the coronavirus pandemic that both Nick and I, I'm sure, will have a lot to say about next time we speak. And perhaps that is for the best, because the deepest lessons can be learned only in retrospect when the storm has passed. I do recommend you read many of his papers on the topic of existential risk, including the technical report titled Global Catastrophic Risks Survey that he co authored with Anders Sandberg. For everyone feeling the medical, psychological and financial burden of this crisis, I'm sending love your way. Stay strong. We're in this together. We'll beat this thing. This is the Artificial Intelligence Podcast. If you enjoy it, subscribe on YouTube, review it with five stars on Apple, podcast support on Patreon, or simply connect with me on Twitter lexfridman spelled F R I D M A N. As usual, I'll do one or two minutes of ads now and never any ads in the middle that can break the flow of the conversation. I hope that works for you and doesn't hurt the listening experience. This show is presented by Cash App, the number one finance app in the App Store. When you get it, use Code lexpodcast Cash App lets you send money to friends, buy Bitcoin, and invest in the stock market with as little as $1. Since Cash App does fractional share trading, let me mention that the order execution algorithm that works behind the scenes to create the abstraction of fractional orders is an algorithmic marvel. So big props to the Cash App engineers for solving a hard problem that in the end provides an easy interface that takes a step up to the next layer of abstraction over the stock market, making trading more accessible for new investors and diversification much easier. So again, if you get Cash App from the App Store, Google Play and use the Code Lex podcast, you get $10 and Cash App will also donate $10 to first, an organization that is helping to advance robotics and STEM education for young people around the world. And now, here's my conversation with Nick Bostrom. At the risk of asking the Beatles to Play Yesterday or the Rolling Stones to play satisfaction. Let me ask you the basics. What is the simulation hypothesis?

That we are living in a computer simulation?

What is a computer simulation? How are we supposed to even think about that?

Well, so the hypothesis is meant to be understood in a literal sense. Not that we can kind of metaphorically view the universe as an information processing physical system, but that there is some advanced civilization who built a lot of computers and that what we experience is an effect of what's going on inside one of those computers. So that the world around us, our own brains, everything we see and perceive and think and feel would exist because this computer is running certain programs.

So do you think of this computer as something similar to the computers of today? These deterministic sort of Turing machine type things? Is that what we're supposed to imagine or we're supposed to think of something more like, like a, like a quantum mechanical system? Something much bigger, something much more complicated, something much more mysterious. From our current perspective, the ones we.

Have today would do fine. I mean, bigger, certainly you'd need more, more memory and more processing power. I don't think anything else would be required now. It might well be that they do have additional. Maybe they have quantum computers and other things that would give them even more oomph. Seems kind of plausible, but I don't think it's a necessary assumption in order to get to the conclusion that a technologically mature civilization would be able to create these kinds of computer simulations with conscious beings inside them.

So do you think the simulation hypothesis is an idea that's most useful in philosophy, computer science, physics, Sort of. Where do you see it having valuable kind of starting point in terms of the thought experiment of it.

Is it useful? I guess it's more informative and interesting and maybe important, but it's not designed to be useful for something else.

Okay, interesting, sure. But is it philosophically interesting or is there some kind of implications of computer science and physics?

I think not so much for computer science or physics per se. Certainly it would be of interest in philosophy, I think also to say cosmology or physics in as much as you're interested in the fundamental building blocks of the world and the rules that govern it. If we are in a simulation, there is then the possibility that say, physics at the level where the computer running the simulation could be different from the physics governing phenomena in the simulation. So I think might be interesting from point of view of religion or just for kind of trying to figure out what the heck is going on. So we mentioned the simulation Hypothesis so far, there is also simulation argument, which I tend to make a distinction. So simulation hypothesis. We are living in a computer simulation simulation argument, this argument that tries to show that one of three propositions is true, one of which is the simulation hypothesis. But there are two alternatives in the original simulation argument, which we can get to.

Yeah, let's go there. By the way, confusing terms, because people will, I think, probably naturally think simulation argument equals simulation hypothesis. Just terminology wise. But so simulation hypothesis means that we are living in a simulation. The hypothesis that we're living in a simulation simulation argument has these three complete possibilities that cover all possibilities. So what are they?

Yeah, so it's like a disjunction. It says at least one of these three is true, although it doesn't on its own tell us which one. So the first one is that almost all civilizations at our current stage of technological development go extinct before they reach technological maturity. So there is some great filter that makes it so that basically none of the civilizations throughout maybe vast cosmos will ever get to realize the full potential of technological development.

And this could be, theoretically speaking, this could be because most civilizations kill themselves too eagerly or destroy themselves too eagerly, or it might be super difficult to build a simulation. So the span of time, theoretically it could be both.

Now, I think it looks like we would technologically be able to get there in a time span that is short compared to, say, the lifetime of planets and other sort of astronomical processes.

So your intuition is the build simulation is not.

Well, so there's this interesting concept of technological maturity. It's kind of an interesting concept to have other purposes as well. We can see, even based on our current limited understanding, what some lower bound would be on the capabilities that you could realize by just developing technologies that we already see are possible. So, for example, one of my research fellows here, Eric Drexler, back in the 80s, studied molecular manufacturing. That is, you could analyze, using theoretical tools and computer modeling, the performance of various molecularly precise structures that we didn't then and still don't today have the ability to actually fabricate. But you could say that, well, if we could put these atoms together in this way, then the system would be stable and it would rotate at this speed and have these computational characteristics. And he also outlined some pathways that would enable us to get to this kind of molecularly manufacturing in the fullness of time. And you could do other studies we've done. You could look at the speed at which, say, it would be possible to colonize the galaxy if you had mature technology. We have an Upper limit, which is the speed of light. We have sort of a lower current limit, which is how fast current rockets go. We know we can go faster than that by just, you know, making them bigger and have more fuel and stuff. And you can then start to describe the technological affordances that would exist. Once a civilization has had enough time to develop at least those technologies we already know are possible, then maybe they would discover other new physical phenomena as well that we haven't realized that would enable them to do even more. But at least there is this kind of basic set of capabilities.

Can you just linger on that? How do we jump from molecular manufacturing to deep space exploration to mature technology? What's the connection?

Well, so these would be two examples of technological capability sets that we can have a high degree of confidence are physically possible in our universe, and that a civilization that was allowed to continue to develop its science and technology would eventually attain.

You can intuit, like we can kind of see the set of breakthroughs that are likely to happen. So you can see, like, what did you call it? The technological set.

With computers, maybe it's easiest one is we could just imagine bigger computers using exactly the same parts that we have. So you could kind of scale things that way. Right. But you could also make processors bit faster if you had this molecular nanotechnology that Eric Drexler described. He characterized a kind of crude computer built with these parts that would perform at a million times the human brain while being significantly smaller, the size of a sugar cube. And he made no claim that that's the optimum computing structure, like, for what? You know, you could build faster computers that would be more efficient, but at least you could do that if you had the ability to do things that were atomically precise. So you can then combine these two. You could have this kind of nanomolecular ability to build things atom by atom and then, say, at a spatial scale that would be attainable through space, colonizing technology. You could then start, for example, to characterize a lower bound on the amount of computing power that a technologically mature civilization would have if it could grab resources, you know, planets and so forth, and then use this molecular nanotechnology to optimize them for computing. You'd get a very, very high, lower boundary on the amount of compute.

So, sorry, just to define some terms. So technologically mature civilization is one that took that piece of technology to its lower bound. What is a technological mature civilization?

Well, okay, so that means it's a stronger concept than we really need for the simulation hypothesis. I just think it's interesting in its own right. So it would be the idea that there is some stage of technological development where you've basically maxed out, that you developed all those general purpose, widely useful technologies that could be developed or at least kind of come very close to the 99.9% there or something. So that's an independent question. You can think either that there is such a ceiling, or you might think it just goes. The technology tree just goes on forever.

Where does your sense fall?

I would guess that there is a maximum that you would start to asymptote.

Towards, so new things won't keep springing up. New ceilings.

In terms of basic technological capabilities, I think there is like a finite set of those that can exist in this universe. Moreover, I mean, I wouldn't be that surprised if we actually reached close to that level fairly shortly after we have, say, machine superintelligence. So I don't think it would take millions of years for a human originating civilization to begin to do this. I think it's more likely to happen on historical timescales. But that's an independent speculation from the simulation argument. I mean, for the purpose of the simulation argument, it doesn't really matter whether it goes indefinitely far up or whether there is a ceiling, as long as we know we can at least get to a certain level. And it also doesn't matter whether that's going to happen in 100 years or 5,000 years or 50 million years. Like the timescales really don't make any difference for this list.

Can you linger on that a little bit? There's a big difference between 100 years and 10 million years. So it doesn't really not matter because you just said, does it matter if we jump scales to beyond historical scales? So we described that. So for the simulation argument, sort of. Doesn't it matter that if it takes 10 million years, it gives us a lot more opportunity to destroy civilization in the meantime?

Yeah, well, so it would shift around the probabilities between these three alternatives. That is, if we are very, very far away from being able to create these simulations, if it's like say billions of years into the future, then it's more likely that we will fail ever to get there. There more time for us to kind of go extinct along the way and similarly for other civilizations.

So it is important to think about how hard it is to build a.

Simulation in terms of figuring out which of the disjuncts. But for the simulation argument itself, which is agnostic as to which of these three alternatives is true, yeah. Okay. You don't have to assume the simulation argument would be true, whether or not we thought this could be done in 500 years or it would take 500 million years.

No, for sure, the simulation argument stands. I mean, I'm sure there might be some people who oppose it, but it doesn't matter. I mean, it's. It's very nice. Those three cases cover it. But the fun part is at least not saying what the probabilities are, but kind of thinking about, kind of intuiting, reasoning about, like, what's more likely. What. What are the kind of things that would make some of the arguments less and more so. Like, but let's. Actually, I don't think we went through them. So number one is destroy ourselves before we ever create simulated.

Right. So that's kind of sad. But we have to think not just what might destroy us. I mean, so there could be some, whatever disaster, some meteorites slamming the Earth a few years from now that could destroy us. Right. But you'd have to postulate in order for this first disjunct to be true, that almost all civilizations throughout the cosmos also failed to reach technological maturity.

And the underlying assumption there is that there is likely a very large number of other intelligent civilizations.

Well, if there are. Yeah. Then they would virtually all have to succumb in the same way. I mean, then that leads off another. I guess there are a lot of little digressions that are interesting.

You know, go there, go there. He's dragging us back.

Well, there are these. There is a set of basic questions that always come up in conversations with interesting people, like the Fermi paradox. You could almost define whether a person is interesting, whether at some point the question of the Fermi paradox comes up. Well, so for what it's worth, it looks to me that the universe is very big. In fact, according to the most popular current cosmological theory, is infinitely big. And so then it would follow pretty trivially that it would contain a lot of other civilizations, in fact, infinitely many if you have some local stochasticity. And infinitely many. It's like infinitely many lumps of matter, one next to another. There's kind of random stuff in each one. Then you're going to get all possible outcomes with probability 1, infinitely repeated. So then certainly there would be a lot of extraterrestrials out there. Even short of that, if the universe is very big, that might be a finite but large number. If we were literally the only one. Yeah. Then, of course, if we went extinct, then all of civilizations at our Current stage would have gone extinct before becoming technological material. So then it kind of becomes trivially true that a very high fraction of those went extinct. But if we think there are many, I mean, it's interesting because there are certain things that plausibly could kill us. If you look at existential risks, and it might be a different. The best answer to what would be most likely to kill us might be a different answer than the best answer to the question, if there is something that kills almost everyone, what would that be? Because that would have to be some risk factor that was kind of uniform overall. Possible civilization.

Yeah. So in this, for the, for the sake of this argument, you have to think about not just us, but like every civilization dies out before they create the simulation.

Yeah. Or something very close to everybody.

Okay, so what's number two in the.

Well, so number two is the convergence hypothesis that is that maybe like a lot of some of these civilizations do make it through to technological maturity, but out of those who do get there, they all lose interest in creating these simulations. So they just. They have the capability of doing it, but they choose not to. Not just a few of them decide not to, but, you know, you know, out of a million, you know, maybe not even a single one of them would do it.

And I, I think when you say lose interest, that sounds like unlikely because it's like they get bored or whatever. But it could be so many possibilities within that. I mean, losing interest could be. It could be anything from it being exceptionally difficult to do to fundamentally changing the sort of the fabric of reality if you do it. Ethical concerns, all those kinds of things could be exceptionally strong pressures.

Well, certainly, I mean, yeah, ethical concerns. I mean, not really too difficult to do. I mean, in a sense, that's the first assumption that you get to technological maturity where you would have the ability, using only a tiny fraction of your resources, to create many simulations. So it wouldn't be the case that they would need to spend half of their GDP forever in order to create one simulation. And they had this difficult debate about whether they should invest half of their GDP for this. It would more be like, well, if any little fraction of the civilization feels like doing this at any point during maybe their millions of years of existence, then there would be millions of simulations. But certainly there could be many conceivable reasons for why there would be this convergence, many possible reasons for not running ancestor simulations or other computer simulations, even if you could do so cheaply.

By the way, what's an ancestor simulation?

Well, that would be the type of computer simulation that would contain people like those we think have lived on our planet in the past and like ourselves, in terms of the types of experiences they have and where those simulated people are conscious. So like not just simulated in the same sense that a non player character would be simulated in the current computer game, where it kind of has like an avatar body and then a very simple mechanism that moves it forward or backwards or. But something where the simulated being has a brain, let's say, that's simulated at a sufficient level of granularity that it would have the same subjective experiences as we have.

So where does consciousness fit into this, do you think, simulation? Is there different ways to think about how this can be simulated? Just like you're talking about now, do we have to simulate each brain within the larger simulation? Is it enough to simulate just the brain, just the minds and not the big universe itself? Is there different ways to think about this?

Yeah, I guess there is a kind of premise in the simulation argument rolled in from philosophy of mind that is that it would be possible to create a conscious mind in a computer and that what determines whether some system is conscious or not is not like whether it's built from organic biological neurons, but maybe something like what the structure of the computation is that it implements. So we can discuss that if we want, but I think it would be forward my view that it would be sufficient, say if you had a computation that was identical to the computation in the human brain down to the level of neurons. So if you had a simulation with 100 billion neurons connected in the same way as the human brain, and you then roll that forward with the same kind of synaptic weights and so forth. So you actually had the same behavior coming out of this as a human with that brain would have, then I think that would be conscious. Now it's possible you could also generate consciousness without having that detailed assimilation there. I'm getting more uncertain exactly how much you could simplify or abstract away.

Can you linger on that? What do you mean? I missed where you're placing consciousness in the second.

Well, so if you are a computationalist, you think that what creates consciousness is the implementation of a computation, Just some property.

Emergent property of the computation itself?

Yeah, yeah, you could say that. But then the question is, what's the class of computations such that when they are run, consciousness emerges. So if you just have something that adds one plus one plus one plus one, like a simple computation, you think maybe that's not going to have any consciousness. If on the other hand, the Computation is one like our human brains are performing, where as part of the computation there is a global workspace, a sophisticated attention mechanism, there is self representations of other cognitive processes and a whole lot of other things that possibly would be conscious. And in fact, if it's exactly like ours, I think definitely it would. But exactly how much less than the full computation that the human brain is performing would be required is a little bit, I think, of an open question. He asked another interesting question as well, which is, would it be sufficient to just have, say, the brain, or would you need the environment that's a nice way in order to generate the same kind of experiences that we have. And there is a bunch of stuff we don't know. I mean, if you look at say, current virtual reality environments, one thing that's clear is that we don't have to simulate all details of them all the time in order for say, the human player to have the perception that there is a full reality in there. You can have say, procedurally generated, which might only render a scene when it's actually within the view of the player character. And so similarly, if this environment that we perceive is simulated, it might be that all of the parts that come into our view are rendered at any given time. And a lot of aspects that never come into view, say the details of this microphone I'm talking into, exactly what each atom is doing at any given point in time might not be part of the simulation, only a more coarse grained representation.

So that to me is actually from an engineering perspective, why the simulation hypothesis is really interesting to think about is how difficult is it to fake sort of in a virtual reality context, I don't know if fake is the right word. But to construct a reality that is sufficiently real to us to be, to be immersive in the way that the physical world is, I think that's actually probably an answerable question of psychology, of computer science, of how, where is the line where it becomes so immersive that you don't want to leave that world?

Yeah. Or that you don't realize while you're in it that it is a virtual world.

Yeah, those are two actually questions. Yours is the more sort of the good question about the realism, but mine from my perspective, what's interesting is it doesn't have to be real, but how can we construct a world that we wouldn't want to leave?

Yeah, I mean, I think that might be too low a bar. I mean, if you think, say when people first had Pong or something like that, I'm sure There were people who wanted to keep playing it for a long time because it was fun and they wanted to be in this little world. I'm not sure we would say it's immersive. I mean, I guess in some sense it is, but like an absorbing activity doesn't even have to be.

But they left that world though. That's the. So like, I think that bar is deceivingly high. So they eventually, so they, you can play Pong or Starcraft or would have more sophisticated games for hours, for months, you know, wow. World of Warcraft could be in a big addiction, but eventually they escape that.

So you mean when it's absorbing enough that you would spend your entire, you would choose to spend your entire life.

In there and then thereby changing the concept of what reality is, because your reality, your, your reality becomes the game, not because you're fooled, but because you've made that choice.

Yeah, and it may be different people might have different preferences regarding that. Some might, even if you had any perfect virtual reality, might still prefer not to spend the rest of their lives there. I mean, in philosophy there's this experience machine thought experiment. Have you come across this? So Robert Nozick had this thought experiment where you imagine some crazy super duper neuroscientists of the future have created a machine that could give you any experience you want if you step in there. And for the rest of your life you can kind of pre program it in different ways so your fondest dreams could come true. You could whatever you dream you want to be a, a great artist, a great lover, have a wonderful life, all of these things, if you step into the experience machine, will be your experiences constantly happy, but you would kind of disconnect from the rest of reality and you would float there in a tank. And so Nozick thought that most people would choose not to enter the experience machine. Mean many might want to go there for a holiday, but they wouldn't want to check out of existence permanently. And so he thought that was an argument against certain views of value. According to what we value is a function of what we experience. Because in the experience machine, you can have any experience you want, and yet many people would think that would not be much value. So therefore what we value depends on other things than what we experience.

So, okay, can you, can you take that argument further? What about the fact that maybe what we value is the up and down of life?

So you could have up and downs in the experience machine, Right, but what can't you have in the experience machine? Well, I mean, that then becomes an interesting question to explore. But, for example, real connection with other people. If the experience machine is a solo machine where it's only you, like, that's something you wouldn't have there. You would have this subjective experience that would be like fake people. But if you gave somebody flowers, that wouldn't be anybody there who actually got happy. It would just be a little simulation of somebody smiling. But the simulation would not be the kind of simulation I'm talking about in the simulation argument where the simulated creature is conscious. It would just be a kind of smiley face that would look perfectly real to you.

So we're now drawing a distinction between appear to be perfectly real and actually being real.

Yeah, so that could be one thing. I mean, like a big impact on history, maybe. It's also something you won't have if you check into this experience machine. So some people might actually feel, the life I want to have for me is one where I have a big positive impact on how history unfolds. So you could kind of explore these different possible explanations for why it is. You wouldn't want to go into the experience Machine if that's what you feel. And one interesting observation regarding this Nozick thought experiment and the conclusions he wanted to draw from it is how much is a kind of a status quo effect. So a lot of people might not want to get this on current reality to plug into this dream machine, but if they instead were told, well, what you've experienced up to this point was a dream, now do you want to disconnect from this and enter the real world when you have no idea maybe, what the real world is? Or maybe you could say, well, you're actually a farmer in Peru growing peanuts, and you could live for the rest of your life in this. Or would you want to continue your dream life as Alex Friedman, going around the world making podcasts and doing research? So if the status quo was that they were actually in the experience machine, I think a lot of people might then prefer to live the life that they are familiar with rather than sort of bail out into.

So that's interesting. The change itself, the leap, whatever.

So it might not be so much the reality itself that we are after, but it's more that we are maybe involved in certain projects and relationships.