Stephen Wolfram on Observer Theory

Okay. Hello, everyone. Well, yesterday I posted an essay about observer theory that's kind of a convergence of a lot of things I've thought about for the last several years. And it's kind of a core piece of the framework, sort of the intellectual framework for the things we've done physics project, the RULIAD multicomputation, bunch of other kinds of applications of that. So I thought what I would do here is tell you what I said in this essay that I posted. So the subject is observer theory, and the first part is called the concept of the observer. We call it perception, we call it measurement, we call it analysis, but in the end, it's about how we take the world as it is and derive from it the impression of it that we have in our minds. We might have thought that we could do science purely objectively, without any reference to observers or their nature, but. But what we've discovered, particularly dramatically in our physics project, is that the nature of us as observers is critical, even in determining the most fundamental laws we attribute to the universe. But what ultimately does an observer say, like us do, and how can we make a theoretical framework for it? Much as we have a general model for the process of computation instantiated by something like a Turing machine, we'd like to have a general model for the process of observation, a general observer theory. Central to what we think of as an observer is the notion that the observer will take the raw complexity of the world and extract from it some reduced representation suitable for a finite mind. There might be zillions of photons impinging on our eyes, but all we extract is the arrangement of objects in a visual scene. Or there might be zillions of gas molecules impinging on a piston. Yet all we extract is the overall pressure of the gas. In the end, we can think of it as fundamentally being about equivalencing. There are immense numbers of different individual configurations for the photons or the gas molecules that are all treated as equivalent by an observer who's just picking out the particular features needed for some reduced representation. There's, in a sense, a certain duality between computation and observation. In computation, one's generating new states of a system. In observation, one's equivalencing together different states. That equivalencing must, in the end, be implemented underneath by computation. But in observer theory, what we want to do is just characterize the equivalencing that's achieved for us as observers. It might, in practice, be all about how our senses work, what our biological or cultural nature is or what technological devices or structures we've built. But what makes a coherent concept of observer theory possible is that it seem to be general, abstract characterizations that capture the essence of different kinds of observers. It's not immediately obvious that anything suitable for a finite mind could ever be extracted from the complexity of the world. Indeed, the principle of computational equivalence implies that computational irreducibility and its multi computational generalization will be ubiquitous. But within computational irreducibility, there must always be slices of computational reducibility. And it's these slices of reducibility that an observer must try to pick out, and that ultimately make it possible for a finite mind to develop a useful narrative about what happens in the world that allows it to make decisions, predictions, and so on. How special is what an observer does? At its core, it's just about taking a large set of possible inputs and returning a much smaller set of possible outputs. And certainly that's a conceptual idea that's appeared in many fields under many different names. Contractive mapping, reduction to canonical form, a classifier, an acceptor, a forgetful functor evolving to an attractor, extracting statistics, model fitting, lossy compression, projection, phase transitions, renormalization, group transformations, and so on. But here we want to think not about what's mathematically describable, but instead about what in general is actually implemented, say, by our senses, or our measuring devices, or our ways of analyzing things at an ultimate level. Everything that happens can be thought of as being captured by the ruliad, the unique object that emerges at the entangled limit of all possible computations. And a vast generalization of ideas like that our brains, like any other material thing, are made of. It's a vast generalization of ideas like that our brains, like any other material thing, are made of atoms. So too, any observer must be embedded as some kind of structure within the ruliad. But a key concept of observer theory is that it's possible to make conclusions about an observer's impression of the world just by knowing about the capabilities and assumptions of the observer, without knowing in detail what the observer is like inside. And so it is, for example, in our physics project, we seem to be able to derive essentially from the structure of the ruliad the core laws of 20th century physics, general relativity, quantum mechanics, the second law, just on the basis of two features of us as observers. That we're computationally bounded and that we believe we're persistent in time, even though underneath, we're made of different atoms of space at every successive moment. And we can expect that if we were to take, if we were to include other features of us as observers, for example, that we believe we are persistent objects in the world, or that we believe we have free will, then we'd be able to derive more aspects of the universe as we experience it, or of natural laws that we attribute to it. But the notion of observers and observer theory isn't purely limited to physical observers. It applies whenever we try to get an impression of something. And so, for example, we can also operate as mathematical observers, sampling the ruliad to build up conclusions about mathematical laws. Some features of us as physical observers, like the computational boundedness associated with the finiteness of our minds, inevitably carry over to us as mathematical observers. But other features do not. But the point of observer theory is to provide a general framework in which we can characterize observers and then see the consequences of those characterizations for the impressions or conclusions observers will form. So now we'll talk about the operation of observers. As humans, we have senses like sight, hearing, touch, taste, smell, and balance. And through our technology, we also have access to a few thousand other kinds of measurements. So how basically do all these work? The vast majority, in effect, aggregate a large number of small inputs to generate some kind of average output, which in the case of measurement, is often specified as a real number. In a few cases, however, there's instead a discrete choice between outputs that's made on the basis of whether the total input exceeds a threshold. Think distributed consensus schemes, weighing balances, and so on. But in all cases, what's fundamentally happening is that lots of different input configurations are all being equivalence, or more operationally, the dynamics of the system essentially make all equivalence states evolve to the same attractor state. So, as an example, let's consider measuring the pressure of a gas. There are various ways to do this, but a direct one is to have a piston and just see how much force is exerted by the gas on this piston. So where does the force come from? At the lowest level, it's the result of individual molecules bouncing off the surface of the piston, each transferring a tiny amount of momentum to it. If we looked at the piston at an atomic scale, we'd see it temporarily deform from each molecular impact. But the crucial point is that at a large scale, the piston moves together as a single rigid object, aggregating the effects of all those individual molecular impacts. But why does it work that way? Essentially, it's because the intermolecular forces inside the piston are much stronger than the forces associated with molecules in the gas. Or put more abstractly, there's more coupling and coherence inside the observer than between the observer and what it's observing. We see that same pattern over and over again. There's some form of transduction that couples the individual elements of what's being observed to. To the observer. Then, within the observer, there's something that, in essence, aggregates all those small effects. Sometimes that aggregation is directly numerical, as in the addition of lots of small momentum transfers. But sometimes it's instead more explicitly like evolution to one attractor rather than another. So consider, for example, the case of vision. An array of photons fall on the photoreceptor cells in our retinas, generating electrical signals transmitted through nerve fibers to our brains. Within the brain, there's then, effectively a neural net that evolves to different attractors depending on what one's looking at. Most of the time, a small change in input image won't affect what attractor one evolves to. But much like with a weighing balance, there's an edge at which even a small change can lead to a different output. One can go through lots of different types of sensory systems and measuring devices. But the basic outline seems to always be the same. First, there's a coupling between what is being sensed or measured and the thing that's doing the sensing or measuring. Quite often, that coupling involves transducing from one physical form to another, Say, from light to electricity or from force to position. Sometimes, then, the crucial step of equivalencing different detailed inputs is achieved by simple numerical aggregation, most often by accumulation of objects, atoms, raindrops, et cetera, or by physical effects, forces, currents, and so on. But sometimes the equivalencing is instead achieved by more obviously dynamical process. It could amount to simple amplification in which, say, the presence of a small element of input, say an individual particle, tips over some metastable system so that it goes into a certain final state. Or it could be more like a neural net, where there's a more complicated translation defined by hard to describe borders between basins of attraction leading to different attractors. But, okay, so what's the end point of a process of observation? Ultimately, for us humans, it's an impression created in our minds. Of course, that gets into lots of slippery philosophical issues. Yes, each of us has an inner experience of what's going on in our mind. But anything else is ultimately an extrapolation. We make the assumption that other human minds also see what we see, but we can never feel it from the inside. We can, of course, make increasingly detailed measurements, say, of neural activity to see how similar what's going on is between one brain and another. But as soon as there's the slightest structural or situational difference between the brains, we really can't say exactly how their impressions will compare. But for our purposes, in constructing a general observer theory, we're basically going to make the assumption, or in effect, philosophical approximation, that whenever a system does enough equivalencing, that's tantamount to it acting like an observer, because it can then act as a front end that takes the incoherent complexity of the world and collimates it to the point where a mind will derive a definite impression from it. Of course, there's still a lot of subtlety here. There has to be just enough equivalencing and not too much. For example, if all inputs were always equivalent to the same output, there'd be nothing useful observed. And in the end, there's somehow got to be some kind of match between the compression of input achieved by the equivalencing and. And the capacity of the mind that's ultimately deriving an impression from it. A crucial feature of anything that can reasonably be called a mind is that something's got to be going on in there. It can't be, for example, that the internal state of the system is fixed. There has to be some internal dynamics, some computational process that we can identify as the ongoing operation of the mind. At an informational level, we might say that there has to be more information processing going on inside than there is flow of information from the outside. Or, in other words, if we're going to be meaningful observers like us, we can't just be bombarded by input we don't process. We have to have some capability to think about what we're seeing. All of this comes back to the idea that a crucial feature of us as observers is that we are computationally bounded. We do computation. That's why we can have an inner sense of things going on. But the amount of computation we do is tiny compared to the computation going on in the world around us. Our experience represents a heavily filtered version of what's happening outside. And the essence of being an observer like this is that we're effectively doing lots of equivalencing to get that filtered version. But can we imagine a future in which we expand our minds or perhaps encounter some alien intelligence with a fundamentally less constrained mind? Well, at some point there's an issue with this, because in a sense, the idea that we have a coherent existence relies on us having limited minds. For without Such constraints, there wouldn't be a coherent self that we can identify with coherent inner experience. Well, let's say we're showing some system, say in nature, from the outside. Can we tell if there's an observer in there? Ultimately not, because in a sense we have to be inside that observer and be able to experience the impression of the world that it's getting. But in much the same way as we extrapolate to believing that, say, other human minds are experiencing things like we're experiencing, so also we can potentially extrapolate to, say, what we might think of as an observer. And the core idea seems to be that an observer should be a subsystem whose internal states are affected by the rest of the system. But where many external states lead to the same internal state, and where there is rich dynamics within the observer, that an effect operates only on its internal states. Ultimately, following the principle of computational equivalence, both the outside and the inside of the observer subsystem can be expected to be equivalent in the computations they're performing. But the point is that the coupling from outside the subsystem to inside, effectively coarse grains, what's outside, so that the inner computation is operating on a much reduced set of elements. Why should any such observer subsystems exist? Presumably, at some level it's inevitable from the presence of pockets of computational reducibility within arbitrarily computationally irreducible systems. But more important for us is that our very existence and the possibility of our coherent inner experience depends on us operating as observers. And almost as a self fulfilling prophecy, our behavior tends to perpetuate our ability to successfully do this. For example, we can think of us as choosing to put ourselves in situations and environments where we can predict what's going to happen well enough to survive as observers. At a mundane practical level, we might do this by not living in places subject to unpredictable natural forces, or doing things like building ourselves structures that shelter us from those forces. Well, we've talked about observers operating by compressing the complexities of the world to inner impressions suitable for finite minds. And in typical situations that we describe as perception and measurement. The main way this happens is by fairly direct equivalencing of different states. But in a sense, there's a higher level story that relies on formalization and in essence, computation. And that's what we usually call analysis. Let's say we have some intricate structure, perhaps some nested fractal pattern. A direct rendering of all the pixels in this pattern ultimately won't be something well suited for a finite mind. But if we gave rules or a program for generating the pattern, we'd have a much more succinct representation of it. But now there's a problem with computational irreducibility. Yes, the rules determine the pattern, but to get from those rules to the actual pattern can require an irreducible amount of computation. And to reverse engineer the pattern to find the rules can require even more computation. Yes, there are particular cases, like repetitive and simple nested patterns, where there's enough immediate computational reducibility that a computational bounded system or observer can fairly easily do the analysis and get the compression. But in general, it's hard, and indeed, in a sense, it's the whole mission of science to pick away at the problem and try to find more ways to reduce the complexities of the world to human level narratives. Computational irreducibility limits the extent to which this can be successful. But the inevitable existence of pockets of reducibility, even within computationally computational irreducibility, guarantees that progress can always, in principle, be made. As we invent more kinds of measuring devices, we can extend our domain as observers. And the same is true when we invent more methods of analysis or identify more principles in science. But the overall picture remains the same. What's crucial to being an observer is equivalencing many states of the world, either through perceiving or measuring only specific aspects of them, or through identifying simplified narratives that capture them. In effect, persistence and measurement tend to do lossy compression. Analysis is more about lossless compression, where the equivalencing is effectively not between possible inputs, but between possible generative rules. All right, now let's talk about how observers construct their perceived reality. Our view of the world is ultimately determined by what we observe of it. We take what's out there in the world and in effect, construct our perceived reality by our operation as observers. Or in other words, insofar as we have a narrative about what's going on in the world. That's something that comes from our operation as observers and in fact, from our physics project. We're led to an extreme version of this in which what's out there in the world is just the whole ruliad. The and in effect, everything specific about our perceived reality must come from how we operate as observers and thus how we sample the ruliad. But long before we get to this ultimate level of abstraction, there are lots of ways in which our nature as observers builds our perceived reality. Think about any material substance, like a fluid. Ultimately, it's made up of lots of individual Molecules doing their thing. But observers like us aren't seeing those molecules. Instead, we're aggregating things to the point where we can just describe the system as a fluid that operates according to the narrative defined by the laws of fluid mechanics. But why do things work this way? Ultimately, it's the result of the repeated story of the interplay between underlying computational irreducibility and the computational boundedness of us as observers. At the lowest level, the motion of the molecules is governed by simple rules of mechanics. But the phenomenon of computational irreducibility implies that to work out the detailed consequences of running these rules involves an irreducible amount of computational work, which is something that we, as computationally bounded observers, can't do. And the result of this is that we'll end up describing the detailed behavior of the molecules as just random. Well, I've talked about this at length elsewhere, but this is the fundamental origin of the second law of thermodynamics. But for our purposes here, the important point is that it's what makes observers like us construct the reality of things like fluids. Our computational boundedness as observers makes us unable to trace all the detailed behavior of molecules and leaves us content to describe fluids in terms of the narrative defined by the laws of fluid mechanics. Our physics project implies that it's the same kind of story with physical space. For our physics project, space is ultimately made of a network of relations or connections between discrete atoms of space that's progressively being updated in what ends up being a computationally irreducible way. But we, as computationally bounded observers can't decode all the details of what's happening, and instead, we end up with a simple aggregate narrative that turns out to correspond to continuum space operating according to the laws of general relativity. The way both coherent notions of matter or fluids and space time emerge for us as observers can be thought of as a consequence of the equivalencing we do as observers. In both cases, there's immense and computationally irreducible complexity underneath. But we're ignoring most of that by effectively treating different detailed behaviors as equivalent, so that in the end, we get to a comparatively simple narrative more suitable for our finite minds. But we should emphasize that what's really going on in the system is something much more complicated. It's just that we, as observers aren't paying attention to that. So our perceived reality is much simpler. Okay, but what about quantum mechanics? Well, in a sense, that's an extreme test of our description of how observers work and the extent to which the operation of Observers constructs their perceived reality. So let's talk now about the case of quantum mechanics. In our physics project, the underlying structure hypergraph that represents space and everything in it is progressively being rewritten according to definite rules. But the crucial point is that at any given stage, there can be lots of ways this rewriting can happen. And the result is that there's a whole tree of possible states of the universe that can be generated. So given this, why do we ever think that definite things happen in the universe? Why don't we just think that there's an infinite tree of branching histories for the universe? Well, it has to do with our nature as observers and the equivalencing we do at an immediate level. We can imagine looking at all those different possible branching paths for the evolution of the universe. And the key point is that even though they come from different paths of history, two states can just be the same. Sometimes it'll be obvious that they're the same. Sometimes we might have to determine, say, whether two hypergraphs are isomorphic. But the point is that to any observer, at least one that isn't managing to look at arbitrary implementation details, the states will inevitably be considered equivalent. But now there's a bigger point. Even though from the outside, there might be a whole branching and merging multi way graph of histories for the universe, Observers like us can't trace that. And in fact, all we perceive is a single thread of history. Or, said another way, we believe that we have a single thread of experience, something closely related to our belief that despite the changing underlying elements from which we're made, we are somehow persistent in time, at least during the span of our existence. But operationally, how do we go from all those underlying branches of history to our perceived single thread of history? We can think of the states on different threads of history as being related by what we call a branchial graph that joins states that have immediate common ancestors. And in the limit of many threads, we can think of these different states as being laid out in branchial space. In traditional quantum mechanics terms, this layout defines a map of quantum entanglements, with each piece of common ancestry representing an entanglement between states in physical space. Whether we're looking at molecules in a fluid or atoms of space, we can think of us operating as observers who are physically large enough to span many underlying discrete elements, so that what we end up observing is just some kind of aggregate average result. And it's very much the same kind of thing in branchial space. We as observers tend to be large enough in branchial Space to be spread across an immense number of branches of history, so that what we observe is just an aggregate, just aggregate average results across all those branches. Well, there's lots of detailed complexity in what happens on different branches, Just like there is in what happens to different molecules or different atoms of space. And the reason that there's inevitably computational irreducibility, or in this case, more accurately, multi computational irreducibility. But computationally bounded observers like us Just perceive aggregate results that average out the underlying apparent randomness to give a consistent thread of experience. And effectively, this is what happens in the transition from quantum to classical behavior. Even though there are many possible detailed quantum threads of history that an object can follow, what we perceive corresponds to a single consistent aggregate classical sequence of behavior. And this is typically true Even at the level of our typical observations of molecules and chemical processes. Yes, there are many possible threads of history for, say, a water molecule. But most of our observations aggregate things to the point where we can talk about a definite shape for the molecule with definite chemical bonds and so on. But there is a special situation that actually looms large in typical discussions of quantum mechanics. We can think of it as the result of doing measurements that aren't aggregating threads of history to get an average, but are instead doing something more like a weighing balance, Always tipping one way or the other. In the language of quantum computing, we might say that we're arranging things to be able to measure a single qubit. In terms of equivalencing of states, we might say that we're equivalencing Lots of underlying states to specific canonical states like spin up and spin down. Why do we get one outcome rather than another? Ultimately, we can think of it all as depending on the details of us as observers. To see this, let's start from the corresponding question of physical space. We might ask why we observe some particular thing happening. Well, in our physics project, everything about what happens is deterministic, but there's still the arbitrariness of where we are in physical space. We'll always see basically the same laws of physics, but the particulars of what we'll observe Depends on where we are, say, on the surface of the earth versus an interstellar space and so on. Is there a theory for where we are? In some sense, yes, because we can go back and see why all the molecules that make us up Landed in the particular place where they did. But what we can't have is an external theory for is just which molecules end up making up us as we experience ourselves. From inside. In our view of physics and the universe, it's in some sense the only ultimately subjective thing where our internal experience is situated. But the point is that basically, even though it's much less familiar, the same thing is going on at the level of quantum mechanics. Just as we happen to be at a certain place in physical space, so we're at a certain place in branchial space. Looking back, we can trace how we got here, but there's no a priori way to determine where our particular experience will be situated. And that means we can't know what the local branchial environment will be. And so, for example, what the outcome of balance like measurements will be. Just as in traditional discussions of quantum mechanics, the mechanics of doing the measurement, which we can think of as effectively equivalencing many underlying branches of history, will have an effect on subsequent behavior and subsequent measurements. But let's say we just look at the level of the underlying multi way graph, or more specifically the multi way causal graph that records causal connections between different updating events. Then we can identify a complicated web of interdependence between events that are timelike, space like and branch like separated. And this interdependence seems to correspond to precisely to what's expected from quantum mechanics. In other words, even though the multi way graph is completely determined, the arbitrariness of where the observer is, particularly in branchial space, combined with the inevitable interdependence of different aspects of the multi way causal graph, seems sufficient to reproduce the not quite purely probabilistic features of quantum mechanics. In making observations of physical space, it's common to make a measurement at one place or time, then make another measurement at another place or time, and for example, see how they're related. But in actually doing this, the observer will have to move from one place to another and persist from one time to another. And in the abstract, it's not obvious that that's possible. For example, it could be that an observer won't be able to move without changing. Or in other words, that pure motion won't be possible for an observer. But in effect, this is something we as observers assume about ourselves. And indeed, as I've talked about elsewhere, actually this is a crucial part of why we perceive space time to operate according to the laws of physics we know. But what about in branchial space? We have much less intuition for this than for physical space, but we still effectively believe that pure motion is possible. For us as observers in branchial space. It could be like an observer in physical space near Say near a space time singularity that an observer would get shredded when trying to move in branchial space. But our belief is that typically nothing like that happens. At some level, being at different locations in branchial space presumably corresponds to picking different bases for our quantum states. Or effectively to defining our measurements differently. And somehow our belief in the possibility of pure motion in branchial space seems related to our belief in the possibility of making arbitrary sequences of making arbitrary sequences of choices in sets of experiments we do. All right, let's talk about observers of abstract worlds. We might have thought that the only thing ultimately out there for us to observe would be our physical universe. But actually they're important situations where we're essentially operating not as observers of our familiar physical universe, but instead of what amount to abstract universes. And what we'll see is that the ideas of observer theory seem to apply there too. Except that now what we're picking out and reducing to internal impressions are features not of the physical world, but of abstract worlds. Our physics project, in a sense, brings ideas about the physical and abstract worlds closer. And the concept of the ruliad ultimately leads to a deep unification between them. For what we now imagine is that the physical universe as we perceive it, is just the result of the particular kinds of sampling of the ruliad made by us as certain kinds of observers. And the point is that we, as observers, can make other kinds of samplings leading to what we can describe as abstract universes. And one particularly prominent example of this is mathematics, or rather metamathematics. Imagine starting from all possible axioms for mathematics, then constructing the network of all possible theorems that can be derived from them. We can consider this as forming a kind of metamathematical universe. And the particular mathematics that some mathematician might study. We can then think of as the result of a mathematical observer observing that metamathematical universe. There are both close analogies and differences between this and the experience of a physical observer. In the physical universe. Both ultimately correspond to samplings of the ruliad, but somewhat different ones. In our physics project, we imagine that physical space and everything in it is ultimately made up of discrete elements that we identify as atoms of space. But in the ruliad in general, we can think of everything as being made up of pure atoms of existence that we call eames. In the particular case of physics, we interpret these eames as atoms of space. But in metamathematics, we can think of eames as corresponding to sub axiomatic elements of symbolic structures. From which things like axioms or Theorems can be constructed. A central feature of our interaction with the ruliad for physics is that observers like us don't track the detailed behavior of all the various atoms of space. Instead, we equivalence things to the point where we can get descriptions that are reduced enough to fit in our minds. And something similar is going on in mathematics. We don't track all the individual sub axiomatic Eames, or usually in practice, even the details of fully formalized axioms and theorems. Instead, mathematics typically operates at a much higher level and more human level, dealing not with questions like how real numbers can be built from Eames or even axioms, but rather with what can be deduced about the properties of mathematical objects like real numbers in a physics analogy to the behavior of the gas. Typical human mathematics operates not at the molecular level of individual eames or even axioms, but rather at the fluid dynamics level of human accessible mathematical concepts. In effect, therefore, a mathematician is operating as an observer who equivalences many detailed configurations, ultimately of eames in order to form a higher level mathematical constructs suitable for our computationally bounded minds. And while at the outset one might have imagined that anything in the ruliad could serve as a possible mathematics, the point is that observers like us can only sample the ruliad in particular ways, leading to only particular possible forms for human accessible mathematics. It's a very similar story to the one we've encountered many times in thinking about physics. In studying gases, for example, we could imagine all sorts of theories based on tracking detailed molecular motions. But for observers like us, with our computational boundedness, we inevitably end up with things like the second law of thermodynamics, the laws of fluid mechanics. And in mathematics, the main thing we end up with is higher level mathematics, mathematics that we can do directly in terms of typical textbook concepts, rather than constantly having to drill down to the level of atoms or eames. In physics, we're usually particularly concerned with issues like predicting how things will evolve through time. In mathematics, it's more about accumulating what can be considered true. And indeed, we can think of an idealized mathematician as going through the ruliad and collecting in their minds a bag of theorems or axioms that they consider to be true. And given such a collection, they can essentially follow the entailment paths defined by computations of the ruliad to find more theorems to add to their bag. And yes, if they put in a false theorem, then because a false premise and the standard setup of logic implies everything, they'll end up with an Infinite explosion of theorems that won't fit in a finite mind. In observing the physical universe, we talk about our different possible senses, like vision, hearing, et cetera, or different kinds of measuring devices. In observing the metamathematical universe, the analogy is basically different possible kinds of theories or abstractions, say algebraic versus geometrical versus topological versus categorical, and so on, with new approaches being like new kinds of measuring devices. Particularly when we think in terms of the ruliad, we can expect a certain kind of ultimate unity in the metamathematical universe. But different theories and different abstractions will pick up different aspects of it, just as vision and hearing pick up different aspects of the physical universe. But in a sense, observer theory gives us a global way to talk about this and to characterize what kinds of observations observers like us can make, whether of the physical universe or of the metamathematical one in physics. We've then seen in our physics project how this allows us to find general laws that describe our perception of the physical world and that turn out to reproduce the core known laws of physics. In mathematics, we're not as familiar with the concept of general laws, though the very fact that higher level mathematics is possible is presumably in essence such a law. And perhaps the kinds of regularities seen in areas like category theory are others, as are the inevitable dualities we expect to be able to identify between different fields of mathematics. All these laws ultimately rely on the structure of the ruliad. But the crucial point is that they're not talking about the raw ruliad. Instead, they're talking about just certain samplings of the ruliad that can be done by observers like us, and that lead to certain kinds of internal impressions in terms of which these laws can be stated. Mathematics represents a certain kind of abstract setup that's been studied in a particularly detailed way over the centuries. But it's not the only kind of abstract setup we can imagine. And indeed, there's an even much more familiar one, the use of concepts and words in human thinking and language. We might imagine that at some time in the distant past, our forebears could signify, say, rocks only by pointing at individual ones. But then there emerged the general notion of rock, captured by a word for rock. And once again, this is a story of observers and equivalences. When we look at a rock, it presumably produces all sorts of detailed patterns of neuron firings in our brains, different for each particular rock. But somehow, presumably, essentially through evolution to an attractor in the neural net in our brains, we equivalents all those patterns to extract our inner impression of the concept of a rock. In the typical tradition of quantitative science, we tend to be interested in doing measurements that lead to things like numerical results. But in representing the world using language, we tend to be interested instead in creating symbolic structures that involve collections of discrete words embedded in a grammatical framework. Such linguistic descriptions don't capture every detail in a typical observer kind of way. They broadly equivalence many things and, in a sense, reduce the complexity of the world to a description in terms of a limited number of discrete words and linguistic forms. Within any person's given person's brain, there'll be thoughts defined by patterns of neuron firings. And the crucial role of language is to provide a way to robustly package up those thoughts and, for example, represent them with discrete words so that they can be communicated to another person and unpacked in that person's brain to produce neuron firings that reproduce what amounts to those same thoughts. When we're dealing with something like a numerical measurement, we might imagine that it could have some kind of absolute interpretation. But words are much more obviously an arbitrary basis for communication. We could pick a different specific word, say, from a different human language, but still communicate the same thing. All that's required is that everyone who's using the word agrees on its meaning, and presumably that normally happens because of shared social history between people who use a given word. It's worth pointing out that for this to work, There has to be a certain separation of scales. The collective impression of the meaning of a word may change over time, but that change has to be slow compared to the rate at which the word is used in actual communication. In effect, the meaning of a word as we humans might understand it, emerges from the aggregation of many individual uses. In the abstract, there might not be any reason to think that there'd be a way to understand words consistently, but it's a story very much like what we've encountered in both physics and mathematics. Even though there are lots of complicated individual details underneath, we, as observers manage to pick out features that are simple enough for us to understand. In the case of molecules in a gas, that might be the overall pressure of a gas. In the case of words, it's a stable notion of meaning. Put another way, the possibility of language is another example of observer theory at work. Inside our brains, there are all sorts of complicated neuron firings, but somehow these can be packaged up into things like words that form human level narratives. There's a certain complicated feedback loop between the world as we experience it and the words we use to describe it. We invent words for things that we commonly encounter, chair, table, and so on. Yet once we have a word for something, we're more able to form thoughts about it or communicate about it, and that in turn makes us more likely to put instances of it in our environment. In other words, we tend to build our environment so that we have ways of making narratives about it that work well, or in effect, so that our inner description of it can be as simple as possible and that it can be as predictable to us as possible. We can view our experience of physics and mathematics as being the result of us acting as physical observers and mathematical observers. Now we're viewing our experience of the conceptual universe as being the result of us acting as conceptual observers. But what's crucial is that in all these cases, we have the same intrinsic features as observers. Computational boundedness and a belief in persistence. The computational boundedness is what makes us equivalence things to the point where we can have symbolic descriptions of the world, for example, in terms of words. And the belief in persistence is what lets those words have persistent meanings. Actually, these ideas extend beyond just language to paradigms and general ways of thinking about things. When we define a word, we're in effect defining an abstraction for a class of things. And paradigms are somehow a generalization of this ways of taking lots of specifics and coming up with a uniform framework for them. When we do this, we're in effect making a classic observer theory move and equivalencing lots of different things to produce an internal impression that's simple enough to fit in our finite minds. In the end, it's all just the ruliad. Our tendency as observers is always to believe that we can separate our inner experience from what's going on in the outside world. But in the end, everything is just part of the ruliad. And at the level of the ruliad, we as observers are ultimately made of the same stuff as everything else. But we can imagine. Can we imagine that we can point at one part of the ruliad and say, that's an observer, and in another part and say, that's not, at least to some extent? The answer is presumably yes, at least if we restrict ourselves to observers like us. But it's a somewhat subtle and seemingly circular story. For example, one core feature of observers like us is that we have a certain persistence, or at least we believe we have a certain persistence. But inevitably, at the level of the raw ruliad, we're continually being made of different atoms of existence that is different aims. So in what sense are we persistent? Well, the point is that an observer coin equivalents those different successive patterns of Eames, so that what they observe is persistent. And, yes, this is, at least on the face of it, circular. And ultimately, to identify what parts of the ruliad might be persistent enough to be observers, we'll have to ground this circularity in some kind of further assumption. What about the computational boundedness of observers like us, which forces us to do lots of equivalencing? At some level, that equivalencing must be implemented by lots of different states evolving to the same states. But once again, there's circularity, because even to define what we mean by the same states, are isomorphic graphs the same, and so on, we have to be imagining certain equivalent thing. So how do we break out of the circularity? The key is presumably the presence of additional features that define observers like us. And one important class of such features has to do with scale. We're neither tiny nor huge. We involve enough Eames that consistent averages can emerge. Yet we don't involve so many Eames that we span anything but an absolutely tiny part of the whole ruliad. And actually, a lot of our experience is determined by our size as observers. We're large enough that certain equivalencing is inevitable, yet we're small enough that we can reasonably think of there being many choices for where we are. The overall structure of the ruliad is a matter of formal necessity. There's only one possible way for it to be. But there's contingency in our character as observers. And, for example, in a sense, there's a fundamental constant of nature as we perceive it, which is our extent in the ruliad, say, measured in eames and appropriately projected into physical space, branchial space, and so on. And the fact that this extent is small compared to the whole ruliad means that there are many possible observers who can think of existing at different positions in the ruliad, and those different observers will look at the ruliad from different points of view and thus develop different internal impressions of perceived reality. But a crucial fact central to our physics project is that there are certain aspects of that perceived reality that are inevitable for observers like us and that correspond to the core laws of physics. But when it gets to more specific questions, what does the night sky look like from where you are? And so on, different observers will inevitably have different versions of perceived reality. So is there a way to translate from one observer to another? Essentially, that's a story of motion. What happens when an observer that's at one place in the ruliad moves to another place? Inevitably, the observer will be made of different eames if it's at a different place. But will it somehow still be the same? Well, that's a subtle question that depends both on the background structure of the ruliad and the nature of the observer. If the ruliad is too wild, think near spacetime, near a singularity, then the observer will inevitably be shredded as it moves. But computational irreducibility implies a certain overall regularity to most of the ruliad, making pure motion at least conceivable. But to achieve pure motion, the observer still has to be made of something that is somehow robust, essentially some lump of computational reducibility that can predictably survive the underlying background of computational irreducibility in space time. We can identify such lumps with things like black holes and particles like electrons, photons, and so on. And yes, in our models, there's probably considerable commonality between black holes and particles. It's not yet clear quite what the analog is in branchial space, though a very simple example might involve persistence of qubits. And in ruleal space, one kind of analog is the very notion of concepts. For an effect, Concepts, as represented, for example, by words, are the analog of particles in rulial space. They are the robust structures that can move across ruleal space and maintain their identity, carrying the same thoughts to different minds. So what does all this mean for what can constitute an observer in the ruliad? Observers, in effect, leverage computational reducibility to extract simplified features that can fit in finite minds. But observers themselves must also embody computational reducibility in order to maintain their own persistence and the persistence of the features they extract. Or, in other words, observers must, in a sense, always correspond to patches of regularity and in the ruliad. But can any patch of regularity in the ruliad be thought of as an observer? Probably not usefully so, because another feature of observers like us is that we're connected in some kind of collective social framework. Not only do we individually form internal impressions in our minds, but we also communicate these impressions. And indeed, without such communication, we wouldn't, for example, be able to set up things like coherent languages with which to describe things. What do we assume about ourselves? A key implication of our physics project and the concept of the ruliad is that we perceive the universe to be the way we do, because we are the way we are as observers. And the most fundamental aspect of observers like us Is that we're doing lots of equivalencing to reduce the complexity of the world to internal impressions that fit in our minds. But just what kinds of equivalencing are we actually doing? At some level, a lot of that is defined by the things we believe or assume about ourselves and the way we interact with the world. A very central assumption we make is that we're somehow stable observers of a changing outside world. Of course, at some level, we're actually not stable at all. We're made up from different eames whose configurations, whose configuration is changing all the time. But our belief in our own stability and, in effect, our belief in our persistence in time makes us equivalence those configurations. And having done that equivalencing, we perceive the universe to operate in a certain way that turns out to align with laws of physics we know. But actually, there's more than just our assumption of persistence in time. For example, we also have an assumption of persistence in space. We assume that, at least on reasonably short timescales, we're consistently observing the universe from the same place and not, say, continually darting around. The network that represents space is continually changing around us. But we equivalence things so that we can assume that, in a first approximation, we are staying in the same place. Of course, we don't believe that we have to stay exactly the same place in exactly the same place all the time. We believe we're able to move. And here we make what amounts to another assumption of stability. We assume that pure motion is possible for us as observers. In other words, we assume that we can go to different places and still be the same us with the same properties as observers. At the level of the raw ruliad, it's not at all obvious that such an assumption, that such assumptions can be consistently made. But as we discussed before, the fact that for observers like us, they can, at least to a good approximation, do this as a reflection of certain properties of us as observers, in particular of our physical scale being large in terms of atoms of space, but small in terms of the whole universe. Related to our assumption about motion is our assumption that space exists, or that we can treat space as something coherent underneath. There's all sorts of complicated dynamics of changing patterns of eames. But on the timescales at which we experience things, we can equivalence these patterns to allow us to think of space as having a coherent structure. And once again, the fact that we can do this is a consequence of the physical scales associated with us as observers. In particular, the speed of light is fast enough that it Brings information to us from the local region around us in much less time than it takes our brain to process it. And this means that we can equivalence all the different ways in which different pieces of information reach us, and we can consistently just talk about the state of a region of space at a given time. Part of our assumption that we're persistent in time Is that our thread of experience Is at least locally continuous, with no breaks. Yes, we're born and we die, and we also sleep. But we assume that, at least on scales relevant for our ongoing perception of the world, we experience time as something continuous. More than that, we assume that we have just a single thread of experience. Or in other words, that there's always just one us going through time. Of course, even at the level of neurons in our brains, all sorts of activity goes on in parallel. But somehow, in our normal psychological state, we seem to concentrate everything so that inner experience follows just one thread of history on which we can operate in a computationally bounded way and form definite memories and have definite sequences of thoughts. We're not as familiar with branchial spaces with physical space, but presumably our fundamental assumption of stability extends there as well. And when combined with our basic computational boundedness, then becomes inevitable. That, as we talked about before, will conflate different quantum paths of history to give us observers a definite classical thread of inner experience. Beyond stability, Another very important assumption we implicitly make about ourselves Is what amounts to an assumption of independence. We imagine that we can somehow separate ourselves off from everything else. And one aspect of this is that we assume. One aspect of this is we assume we're localized and that most of the ruliad doesn't matter to us, so that we can equivalence all the different states of the rest of the ruliad. But there's also another aspect of independence. In effect, we can choose to do whatever we want independent of the rest of the universe. And this means that we assume that we can, for example, essentially do any possible experiment, make any possible measurement, or go any way we want in physical or branchial space, or indeed rulial space. We assume that we effectively have free will about these things, Determined only by our inner choices and independent of the state of the rest of the universe. Ultimately, of course, we're just part of the ruliad, and everything we do is determined by the structure of the ruliad and our history within it. But we can view our belief of freedom as a reflection of the fact that we don't know a priori where we'll be located in the ruliad and even if we did, computational irreducibility would prevent us from making predictions about what we will do. Beyond our assumptions about our own independence from the rest of the universe, there's also the question of independence between different parts of what we observe. And quite central to our way of processing the world is our typical assumption that we can think about different things separately. In other words, we assume it's possible to factor what we see happening in the universe into independent parts. In science, this manifests itself in the idea that we can do controlled experiments in which we study how something behaves in isolation by from everything else. It's not self evident that this will be possible, and indeed, in some areas, like ethics, it might be fundamentally not be. But we as observers tend to implicitly assume it. And actually we normally go much further because we typically assume that we can describe and think about the world symbolically. In other words, we assume that we can take all the complexity of the world and represent at least the parts of it we care about. In terms of discrete symbolic concepts of the kind that appear in human or computational language. There's lots of detail in the world that our limited collection of symbolic concepts doesn't capture and effectively equivalences out. But the point is that it's this symbolic description that normally seems to form the backbone of the inner narrative we have about the world. There's another implicit assumption that's being made here, however, and that's that there's some kind of stability in the symbolic concepts we're using. Yes, any particular mind might parse the world using a particular set of symbolic concepts, but we make the implicit assumption that there are other minds out there that work like ours. And this makes us imagine that there can be some form of objective reality that's just always out there to be sampled by whatever mind might happen to come along. Not only, therefore, do we assume our own stability as observers, we also assume a certain stability to what we perceive of everything that's out there underneath. There's all the wildness and complexity of the ruliad. But we assume that we can successfully equivalence things to the point where all we perceive is something quite stable and something we can describe as ultimately governed by consistent laws. It could be that every part of the universe just does its own thing with no overall laws tying everything together. But we make the implicit assumption that no, the universe, at least as far as we perceive it, is a more organized and consistent place. And indeed, it's that assumption that makes it feasible for us to operate as observers like us at all, and even to imagine that we can usefully reduce the complexity of the world to something that fits in our finite minds. The cost of observation what resources does it take for an observer to make an observation? In most of traditional science, observation is at best added as an afterthought, and no account is taken of the process by which it occurs. And indeed, for example, in the traditional formalism of quantum mechanics, while measurement can have an effect on a system, it's still assumed to be an indivisible act without any internal process. But in observer theory, we're centrally talking about the process of observation, and so it makes sense to try asking questions about the resources involved in this process. We might start with our own everyday experience. Something happens out in the world. What resources? And, for example, how much time does it take for us to form an impression of it? Let's say that out in the world, a cat either comes into view or it doesn't. There are signals that come to our brain from our eyes, effectively carrying data on each pixel in our visual field. Then, inside our brain, these signals are processed by a succession of layers of neurons, with us in the end concluding either that there is a cap there or there's not. And from artificial neural nets, we can get a pretty good idea of how this likely works. And the key to it, as we talked about before, is that there's an attractor. Lots of different detailed configurations of pixels, all evolved the same CAT or no cat final state. The different configurations have been equivalenced so that only a final conclusion survives. The story is a bit trickier, though, because CAT or no CAT really isn't the final state of our brain. Hopefully, it's not the last thought we have. Instead, our brain will continue to think more thoughts. So CAT or no CAT is at best some kind of intermediate waypoint in our process of thinking, an instantaneous conclusion that we'll continue to build on. Indeed, when we consider measuring devices like a piston measuring the pressure of a gas, we similarly usually imagine that they will come to an instantaneous conclusion, but continue operating and producing more data. But how long should we wait for each individual conclusion? How long, for example, will it take for the stresses generated by a particular pattern molecules hitting a piston to dissipate out and for the piston to be ready to produce more data? There are lots of specific questions of physics here, but if our purpose is to build a formal observer theory, how should we think about such things? There is something of analogy in the formal theory of computation, an actual computational system, say, in the physical world. Will just keep computing. But in formal computation theory, it's useful to talk about computations that halt and about functions that can be evaluated and give a definite answer. So what's the analog of this in observer theory? Instead of general computations, we're interested in computations that effectively implement equivalences. Or put another way, we want computations that destroy information that have many incoming states but few outgoing states. As a practical matter, we can either have the outgoing states explicitly represent whole equivalence classes, or they can just be canonical representation representatives, like in a network where at each step each element takes on whatever the majority or consensus value of its neighbors was. But however it works, we can still ask questions about what the computational resources were that were involved, how many steps did it take, how many elements were involved, and with the idea that observers like us are computationally bounded, we expect limitations on these resources. But with this formal setup, we can start asking just how far an observer like us can get, say, in coming to a conclusion about the results of some computationally irreducible process. An interesting case arises in putative quantum computers. In the model implied by a physics project, such a quantum computer effectively performs many computations in parallel on the separate branches of a multi way system representing the various threat threads of history of the universe. But if the observer tries to come to a conclusion about what actually happened, they have to knit together all those threads of history, in effect by implementing equivalences between them. One could in principle imagine an observer who just follow all the quantum branches, but it wouldn't be an observer like us. Because what seems like a core feature of observers like us is that we believe we have just a single thread of experience. And to maintain that belief that our process of observation must equivalent to all the different quantum branches, how much effort will that be? Well, inevitably, if a thread of history branched, our equivalencing has to undo that branching. And that suggests that the number of elementary equivalencings will have to be at least comparable to the number of elementary branchings, Making it seem that the effort of observation will tend to be at least comparable to reduction. Will tend to be at least comparable to the reduction of effort associated with parallelism in the underlying quantum process. Actually, I have to say I wonder. In general, it's interesting to compare this sort of effort of observation with the effort of computation with our concept of elementary equivalencings. We have a way to measure both in terms of computational operations. And yes, both could in principle be implemented by something like a Turing machine. But in practice, the equivalencings might be most conveniently modeled by something more like string rewriting. And indeed, one can often go much further, talking not directly in terms of equivalence things, but rather about processes that show attractors. There are different kinds of attractors. Sometimes, like in Class 1 cellular automata, there's just a limited number of static fixed global fixed points, say, either all cells black or all cells white. But in other cases like such as Class 3 cellular automata, the number of output states may be smaller than the number of input states. But there may be no computationally simple characterization of them. Observers like us, though, mostly seem to make use of the fixed points. We try to symbolicize the world, taking all the complexities out there and reducing them to discrete conclusions that we might, for example, describe using the discrete words in a language. There's an immediate subtlety associated with attractors of any kind, though. Typical physics is reversible in the sense that any process, say two molecules scattering from each other, can run equally well forwards and backwards. But in an attractor, one goes from lots of possible initial states to a smaller number of attractor final states. There are two basic ways this can happen, even when there's underlying reversibility. First, the systems on studying can be open in the sense that effects can radiate out of the region that one's studying. And second, the states the system gets into can be complicated enough that, say, a computationally bounded observer will inevitably equivalents them. And indeed, that's the main thing that's happening. For example, when a system reaches thermodynamic equilibrium, as described by the second law. And actually, once again, there's often a certain circularity. One is trying to determine whether an observer has finished observing and come to a conclusion. But one needs an observer to make that determination. Can we tell if we finished forming a thought, or we have to think about it, in effect, by forming another thought. Put another way, imagine we.