History of Science and Technology Q&A (December 28, 2022)

Okay. Hello everyone. Welcome to another episode of History of Science and Technology. Q and A. Let's see, we have a number of questions that got saved up here. And. Let's see, there's one from William here. Was the invention of computers inevitable? Will evolution always stumble upon universal computers? Given enough resources, what are the implications of laws of physics and reality? Well, let's talk about the kind of historical aspect of that. So first of all, the idea that it's possible to make a universal computer. That is, it's possible to have a fixed set of underlying instructions that can be knitted together to do any computation one wants. That idea turns out to be surprisingly ubiquitous among formal systems. You might have thought that in order to make a computer that would be able to do arbitrary computations that you would have to have something was set up using something like the machine instructions that an actual microprocessor uses in know things with registers and arithmetic and this and that and the other. But that is just not the case. It is. You know, one of the things that's come out of lots of work that I've done is this idea of the principle of computational equivalence. The idea that above some pretty low threshold, almost any system whose behavior doesn't look obviously simple is actually capable of universal computation. That is that you can take any its primitive components, whether it's some simple rule about black and white cells or whatever else, and by giving appropriate initial conditions, can make it compute anything you want to compute. Now, it may be non trivial to encode the computation to begin with, it may be non trivial to decode the results, but the stuff that happens in the middle can be an arbitrarily sophisticated computation. And that, that is something that can be done with a whole range of different kinds of underlying underlying primitives. So let's talk about the history of kind of the discovery of that idea. It took a surprisingly long time for that idea to emerge and be understood. And I would say that even today, the implications of that idea for, for example, science and philosophy and so on are not yet well understood. I mean, I myself have been sort of exploring those ideas for more than 40 years now and coming up with things like computational irreducibility, which I invented in the mid-1980s. And but computational irreducibility has just immense consequences for the way one thinks about the world. The way one thinks about, you know, philosophical questions like free will, the way one thinks about questions about what natural science really is, all those kinds of things, and what the capabilities of science are, has huge implications. But it's something that has taken a very long time for us to kind of wrap our heads around and understand. We've certainly been helped by the fact that there are actual practical computers in the world, and we can get some experience from seeing how those work. I think that we have kind of, you know, I've developed for myself lots of intuition about sort of how the computational universe works by doing experiments on simple programs and what they do. I mean, it's interesting right now in these times, as we see these large language model AI systems like ChatGPT and so on, able to do all the things they're able to do, able to write essays about almost anything that sound really good, even though if you take them apart, they might be complete nonsense. That is, again, telling one something. It's giving one an intuition about how to think about thinking, how to think about computation. I don't think I've yet fully internalized that intuition. I mean, I think in that particular case, the thing that I'm understanding is that something I've long believed about human thinking is that it's really based on sort of patterns. And we're sort of saying, well, there's this template that fits this thing, fits that thing, and so on. And what's happening as we look at these neural nets and we have sort of larger and larger sort of arcs of training in these neural nets of sort of that what's happening is that we're getting sort of bigger patterns. It's not just the pattern that says in English, you know, Q's are mostly followed by use. It becomes a larger pattern and then a larger pattern and larger pattern. And I think the thing we'll see next is serious reasoning by analogy, which we haven't yet seen in these systems. But I think it will come because that's again, it's kind of a pattern, fits here, fits there, and so on. And I think that becomes kind of the story, and perhaps that is the ultimate story of sort of what's the particularity of human thinking as compared to arbitrary computation? Because we can go out and just sample the computational universe and find all kinds of computations that do all kinds of wild things, where we say we have no human resonance with that computation. We don't know. We as humans can see it's doing all those bits of flapping around as they flap around, but we don't have sort of a human way to understand that. And I think there's some sort of specialization of human thinking that is not the generic arbitrary computation. I mean, a lot of My life has been spent building our full scale computational language, Wolfram Language, which is an attempt to sort of formalize those parts of the computational world that we care about, to formalize them in sort of a precise way. But if we want to characterize what is it about sort of the human thinking sort of situation that is not just generic computation. And by the way, computation may do things that are very valuable to us, but not in a way that works like the way human thinking works. So, you know, there's a question of how do we characterize that? And I think this idea of these kind of templates and templates for templates, et cetera, et cetera, et cetera, I've long assumed, probably for 40 years have assumed that that's kind of at some level how human thinking works. And in some sense that's what I've tried to capture in Wolfram Language and its predecessor SMP, back 40 years, in the last 43 years or so of, you know, you're making transformations for patterns, for symbolic expressions. That's a very formalized version of what I think is sort of going on in the human thinking case. But we now get to see a new intuition about that as we can actually see something that is writing kind of perhaps nonsense in detailed content, but in terms of form, sort of flowing English text. So anyway, we've got sort of different levels of intuition about how, sort of how to think about computation. Another. And so if we look at that historically, the idea that you could take sort of things we think about and formalize them. So there are two kinds of things we can imagine sort of thinking about in terms of computation. One is thinking and the other is nature. I think the idea of thinking about nature in terms of computation is probably an idea that I guess I'm probably the one who really pioneered that idea in the early 1980s and I sort of backed into that idea, but that idea that you can make kind of program based models of nature. But there are these two cases anyway, thinking about human thinking in terms of what we would now call computation, thinking about nature in terms of what we talk about as computation in terms of human thinking. The process of formalizing human thinking has a long history. Aristotle, 5th century BC logic. That was a formalization, that was a first level formalization of human thinking, of saying what's the pattern? In a sense, that's the predecessor of GPT is the patterns that syllogisms made in logic back in antiquity. And then we go many years into the future and we get to people like George Boole in the 1830s and so on, 1840s, talking about the sort of laws of thought and the idea of having sort of a formal, a more mathematical kind of way to think about this process of doing logic. But we skip over there. Gottfried Leibniz in the late 1600s, who had kind of this idea of kind of formalizing thinking. His PhD thesis was about how to solve legal cases using logic. And he had this idea of a characteristica universalis, a universal language in which one would convert all kind of everything that we talk about in terms of human language into this formal language. And then that formal language, kind of in a very Wolf Malpha like style, would turn into something computational where one could just compute who was right and who was wrong in this legal case. That was kind of an idea that Leibniz had in the late 1600s. There was a circle of ideas at that time that had to do with making these sort of formalized languages, so called philosophical languages, which would transcend the details of human language and be something like the thing I now talk about as symbolic discourse language. So it's, you know, Leibniz, I think, was pretty close to the idea of a universal computer, but he didn't quite have it. And in fact it took a long time for the idea of even an abstract function to emerge. The concept of a function F, where you don't say what it is, you know, F is the plus exponentiation, whatever else, that was a thing well understood by the, by the 1700s and people at Euler were using that, that idea over and over again. But the idea of an arbitrary function where you could sort of start talking about things true for an arbitrary function, that was something that really came as the formalization of calculus and so on. Formalization of mathematics sort of got really developed in the late 1800s. Now there had been other sort of previous efforts. So for example, Charles babbage in the 1830s and so on had been trying to make something which would be an automated generator of mathematical tables. So in the mid-1600s it was figured out by, I think a chap called Chicard was the first person we know of who did this. But Pascal worked on this and so on to make kind of a gear based calculator where you're doing arithmetic by essentially patterns of gearing. There was a precursor of that in antiquity, the Antikythera device from around 0 A.D. or something, 1st century B.C. 1st century A.D. we don't really know. We found only one example of this from this one shipwreck off the island of Antikythera. That is a kind of a gear based cog, based, in that particular case, astronomical calculator. But presumably many more of those things existed, even antiquity. And so this idea that you can kind of do arithmetic, do computations, do the motion of astronomical bodies in this sort of gear based way, you know, that motion of astronomical bodies in a gear based way sort of reminds one of the whole epicycles idea of Ptolemy and so on. And that was, in a sense, the thing that was being implemented by the Antikythera device in a simple case. But so this idea that you could take things that were sort of arithmetic like and make them mechanical, that was an idea that probably exists in antiquity, certainly was known by the 1600s. But then the thing that Babbage sort of tried to figure out was, okay, we want to make this machine a big steam powered thing the size of a locomotive. That that would be something that could compute today. It could do a table of algorithms tomorrow. You could have it work out life insurance tables. He wanted something where you could reprogram it to do these different things. And for that he developed this idea of punch cards. Punch cards have been developed around 1800 by Jacquard for doing loom control, for weaving and so on, and sort of automating, taking a weaving pattern and putting it into fabric. So Babbage had that idea. And then Ada Lovelace, who came along as sort of an expositor primarily of Babbage's engineering idea, was, I think, the one who kind of first got the chance to figure out, yes, there's an abstract thing to talk about here that transcends the sort of details and of the engineering structure of this particular system. And, you know, she was talking about the analytical engine, which Babbage's name for this, this device would weave algebraical patterns as the Jacquard loom weaves patterns of flowers and birds, which is kind of a poetic version of talking about sort of this idea of arbitrary computation. And she had figured out that you could sort of compose music with this and you could do algorithmic music compos and a bunch of other things. So this was a kind of take things that we normally do as humans with our sort of processes of thinking and make them sort of formalized and computational. Now, this notion that you could take sort of the question is this idea of universality, what does it mean to be universal? Well, it kind of means you can not just do the first thing you thought about doing, but you can do all these other things. As well. And I think that is, that's the thing where people had a hard time understanding what's the universe of things that you would want to be doing, you know, that you would want to cover in a universal system. And so people started thinking about, well, what about how do we do math universally? You know, David Hilbert, late 1800s, beginning of the 1900s was very big on let's have a formalized version of math where we just put down these primitives and then all of math is just mechanically grindable out from that. And I think that idea, well, I mean there are many twists to the story about sort of the, you know, taking logic and being able to sort of crush down logic to not use and, or not, but just use NAND. That was something realized around 1900, sort of formalized, maybe 1910 by Henry Scheffer and people like that, that that was a thing. And so that again gave this idea there are simpler primitives that you can use that puff out to bigger things. And that led Moses Schoenfinkel in 1920 to come up with combinators. Combinators are the first real full example of what we would now think of as a computational universal system. And they are, they're things that the S and K combinators, you put them, you kind of construct these symbolic expressions and you can use them to do arbitrary computations. Invented in 1920, still today, combinators are super hard to understand for humans. I mean, I wrote a book about combinators a couple of years ago now, and it's an attempt to kind of take into modern times kind of the idea of combinators. And a lot of interesting things can be said about them. But the fact is they're not very human compatible. They are computation in the raw, powerful, but not very human. And I think that, that, so that was an example of the idea of universal computation. It was there in 1920 with even what Schoenfinkel wrote was pretty clear. But it was very hard to really wrap one's head around what was going on with combinators. And it really took, I mean, Godel for Godel's theorem came up with another version of universal computation, general recursive functions. One of the things that happened was Godel wasn't sure he could see that general recursive functions were general enough to cover arithmetic type mathematics. He didn't know whether they were really general, whether they would apply to actual human minds, for example, or whether they were just a thing that was closed within the domain of mathematics. Similarly, when Turing came up with turing machines in 1936. He came up with them as a way to cover sort of the mathematical kind of types of computations that he imagined were the thing we meant by computation. The question of whether they were broad enough to cover brains, for example. Well, 15 years later he thought about that a bunch more in sort of early, early talking about AI and so on. But it was something where it was not self evident, that sort of, that kind of computation, a Turing machine, how, how global it really was, how universal it actually was. And Turing himself, for example, didn't think that Turing machines would cover physics or biology for that matter. And when he worked on those things, he thought about completely other models that were much more traditional mathematical equation based models. I mean, in the end it sort of fell to me to kind of take on the, the idea of well, let's see if we can actually think about modeling physics. I didn't think at the beginning, I didn't think about fundamental physics. Other people had talked about that sometimes in ways that didn't seem very sensible to me. But I kind of got into the can we take the essence of practical physics and the modeling we do in physics and make that kind of computational, so to speak? And that did start working and one did start getting different intuition from that. But so I think the real question is in terms of, you know, how universal computation arises and how people understand what it means and so on. I think it's much more in the, well, how do we use it and how do we get to the point where we have been able to see how it connects with things more so than the actual sort of, you know, Moses Schoenfelnkel understood universal computation, but nobody used what he did even pretty much to today because it was just computation in the raw that was very hard to understand. And it was, you know, the universal computation component, for example, in natural science. I think it's very important, but that's something that has been completely unobvious until, until quite recently. And it's something where it's kind of like you have to kind of see it from the science side before you kind of can understand it from the universal computation side. So in terms of sort of the alternate histories of science, for instance, let's say the Antikythera device had developed further in antiquity and people had gotten to the point that they understood that you could make a universal mathematical computation device in antiquity. I mean, I kind of sometimes have this idea that maybe Archimedes was involved in the kind of Antikythera device construction. There's some evidence for that at least his sort of school of things in Syracuse, Sicily was involved with this. But you know, I kind of have this idea, what if Archimedes had figured out that you could make kind of a thing like Mathematica, so to speak, back in, you know, the, whatever, you know, third century B.C. no, he was later than that, the first century B.C, second century B.C. and you know, how would that have changed the way that we think about science? Well, the answer is we might have been discovering calculus much later. We would have had program based models of the world and we would have been executing those models. Everybody would have understood computational irreducibility hundreds of years ago. People would have had a much different view of what science can achieve of because people have the view science, exact science is about you write down the equation and then kaboom, you've got the answer to everything. This is what computational irreducibility says doesn't work. Even though you might know the rules for a system, there's still sort of this irreducible computational work to see its consequences. And that's a thing that is that, that would have been understood hundreds of years earlier than it, than it has been understood. It's still not widely understood, as widely understood as it should be by, by a long way. So, you know, I think that's, that's a thing that would have changed. Now, you know, is it the case that, well, the idea of using sort of, that nature is full of universal computers. That's an idea that kind of is, you know, something that I introduced. And that idea, I mean I can kind of, you know, could that idea have arisen hundreds of years earlier? I'm not sure. I think that my ability to introduce that idea was a consequence of the fact that I had sort of some ambient understanding of computers, plus I had the ability to actually do computer experiments. And I started doing computer experiments while in the early 70s I did my very first ones, but more seriously at the end of the 1970s, beginning of the 1980s. And really that was in terms of the development of human technology, that was pretty much the first time you could do those in a serious and fluid kind of way. And that's probably what led to the realization that there was this sort of connection between, this fundamental connection between ideas of computation and ideas of natural science. But so that's a very complicated answer to the question that was asked about. So the inevitability of universal computation, the phenomenon of universal computation, the principle of computational equivalence, kind of tells you that's absolutely ubiquitous. The understanding of it and the connection of it to kind of us humans and the way we think about things, that is much more a question of the history of ideas now. You know, it's an interesting observation with the current round of neural networks that, that are increasingly universal neural networks that are kind of useful for lots of kind of human like tasks. The fact that that is possible is telling us something. The fact that that's possible is probably telling us something very deep about the way that human thinking is kind of carved off from the general of. What happens with arbitrary computation. It's telling us that there's this class of computation that is human thinking like computation, that is different from general computation. General computation is bigger than human thinking like computation. I mean, we are fond of thinking that we are the, the coolest things in the universe type thing, but when it comes to sort of our computational abilities, I just don't think that's true. I think, and by the way, a sort of version of this that's emerged in our physics project is if you think about general computation and even multi computation, where you have these many paths of computational processes branching and merging and so on, when you think about those things, there's the question of a human observer, how similar or not are they to arbitrary computation? What we've realized is that human observers or observers like us have these two characteristics that are rather limiting. We're computationally bounded and we believe that we have a persistent thread of experience through time. And those things are specializations of the general computational thing that are for observers like us. And my guess is that what we're learning from the fact that these neural nets can be as successful as they can on so many human like tasks is that we're learning another sort of universal fact about the way in which we are limited relative to the arbitrary computational universe. And you know, what that should lead to is a kind of narrative theory of thinking and so on that is, you know, intermediate between the kind of the level of the neurons clicking on and off and the level of sort of psychology and so on. And you know, I think it's an interesting thing that there's probably there is science to be taken out of that. It's not traditional mathematical science. It's a different type of science that is again, you know, I'm not sure quite how to think about it yet, but it's something where we're getting this idea that human thinking is this sort of pattern based thing and that that really works and it's patterns upon patterns upon patterns and kind of meta patterns of patterns and so on. And one thing that's sort of interesting about that, in, like, Wolfram language, the whole system is based on transformations for patterns. And in some sense, one's building up patterns upon patterns there. But it isn't quite the same way that we're thinking about patterns upon patterns being built by neural nets or by the sort of model of thinking that we have there. So anyway, it's a challenge to understand kind of what might a theory that's based on kind of thinking about thinking in those kinds of terms be like in the history of AI folks like my friend Marvin Minsky, you know, we're talking about things like, you know, how do we think about thinking? I was never very convinced by the things that Marvin had to say about that. I always thought that they were kind of. Well, they're a little bit like, you know, if we take Freud, for example, and, you know, his kind of view of psychology, you know, the ego, the superego, the ID type thing, that's a kind of a narrative story about how brains work. That probably has some legs. But it's not clear, you know, it's not clear how you generalize that. It's not clear how you turn that into a more elaborate theory. One of the things that's been great in the history of physics, for example, is that it's been possible to do a great deal of elaboration. That is, you have a basic model. It allows you to work out some. One particular case, like the pendulum swinging back and forth. But then that same kind of model can be applied to the. The. The triple pendulum. It can be applied to the much more complicated system, et cetera, et cetera, et cetera. There's collaborative tower that gets built in a field like physics. And really what will allow fields to take off is when there are similar kind of elaborative towers that can be built there. And so the question of some of Marvin's theories about AI, which I admit that even though I talked to him a lot, I'm not sure I understood that. Well, theory of frames or theory of whatever else, it's like, so how do you build something? How do you build a tower on top of that? And that's the thing that when one is able to do that, when one's able to sort of reason about reasoning in a kind of way where one could build a tower, that's where I think we start getting mileage from the kinds of things that science is really good at, the kinds of Things that formal science is good at, which are different from the kinds of things that we as just everyday human thinkers are good at. Just a few thoughts about that. Let's see. Zayden is commenting. I don't think that computing technology was possible to be conceived until after the Industrial Revolution. I wonder about that. I mean, the Antikythera device is how big a computer do you want? It is difficult to have a cogs based computer go the full distance because by the time you have a computation that can be of arbitrary length, you need arbitrary force on the first cog to turn the other cogs. And that's a hard thing to achieve mechanically. And so arguably the, you know, one could argue that electronics and particularly electronic amplifiers were necessary so that you didn't have that problem of kind of the, the, the hard to turn cog, because once you have electronic amplifiers, you don't, you can just say there's an electronic signal and oh, we're just going to amplify it to feed it out to all these other things. And, and you know, that's kind of one of the key things for digital electronics is, is one can do things like that. One doesn't have to worry about, oh, the signal is going to get too decayed and it won't be able to drive this other part, it won't be able to last long enough to do the computation and so on. So there's an argument that one couldn't have gotten there with mechanical devices alone, but I think one could have gotten quite a long way. I mean, for example, it's, you know, the logic machines that got built in the 1800s, the, they're not very impressive, but they could have been built in antiquity. Somebody perfectly well could have gone to Plato's academy and said, hey, I've got this thing. It's made of the same kind of metallic object, you know, construction materials that the Antikythera device was made of. And it does logic. And you know, Aristotle might have said wow, or he might have not understood what on earth it was or, or, or a variety of possibilities. But I think, I mean, things that happened in the Industrial revolution, the things like the invention of the punch card, that was an Industrial Revolution kind of thing. And maybe the concept of kind of the autonomous doing of things, that was probably an Industrial revolution concept. Before that time, the sort of motive power was something that came from animals or from the wind or from something else. The concept that you could make motive power from within the system as like in a steam engine or something was an industrial revolution kind of concept, I think. And, you know, it's famously Babbage, when he was talking about mathematical tables and had been involved in some big project to make mathematical tables by hand, was, you know, had this statement, you know, what was it? I, I wish by God this could be done by steam. He said, in other words, that this process of making mathematical tables could be automated. But his version of describing that was done by steam. Steam was the idea that, you know, was, you know, that it was a, was a, it was a, that was, that was kind of his metaphor for automation. Zayden is pointing out that antiquity, in antiquity, the knowledge to build a steam engine sort of existed as well, and commenting that, you know, people were depressingly cheap in those days and that meant that there wasn't a motivation to do automation. You know, a friend of mine named Armandango, who's a classics professor in Oxford, has written about the notion of innovation in antiquity. And I think his main conclusion has been that one of the areas of innovation was in the military area. If you made a siege machine that was better than the other guy's siege machine, you were going to win. If you made, you know, military technology was worth having because it was allowing you to do things that the other side just couldn't do. You throw enough people into it, you know, even if you throw a lot of people into it, you don't get to scale a, you know, a wall, a sheer wall, you know, just with people. It just, you know, there isn't a way to do that. And so that was a driver, I think, for some kind of early technology. I think that in. I do agree that there is plenty that was not automated in antiquity because people were cheap. The same is true today. I mean, there's a lot of the production line for your average smartphone that is done by people because it's still the case that, you know, that sort of, at least in some parts of the world, human labor is cheaper than sort of setting up the automation, the technology to do it. And human labor is more reliable than the technology in certain ways. And so even Today, you know, 2,000 years after antiquity, so to speak, we're still in a situation where we're. The motivation to do, make robotic factories is not as great as it might be because human labor is still cheap enough relative to the amount that the things that are made in robotic factories cost. And the whole supply chains that supply those robotic factories and, you know, where you mine the metal out of the ground and do this and do this and do this and do this. You have to, you have to kind of cheapen that whole supply chain before it gets to the point where you've got to cheapen the actual final assembly process and things like that to the point where you don't have human people in the loop. But I think it is certainly true that the motivation to automate comes in part when human labor gets too expensive to be used for things. Sometimes the automation happens just because it can, and it's not driven by a dire need to avoid the cost of human labor. But sometimes it is. I think the, the, the question of, you know, this is supposed to be about history of science and technology, but let's talk a little bit about the future too. The question of, you know, once we get to the point where we can automate almost everything, you know, where does that leave us humans? I've said this many times that the, you know, the one thing the automation will never do is to figure out what you want the automation to do. There's still a spark of kind of what are the goals that has to come from outside that automated system. But it is an interesting thing to think about as you automate more and more and as more and more of the economy becomes bots negotiating with bots, bots buying things from bots. What does it look like when the vast majority of economic activity is bot to bot activity? What does that look like? And I had a thought yesterday actually, that kind of funny that, you know, when most economic activity is in the bot sphere and where most of it is, is, you know, then at some level maybe what happens is everything that is relevant to humans is free, because it could be the case. And I was thinking, what's an analogy? Well, the answer is to a dog, you know, a domesticated dog, its food is free. It doesn't have to do anything to get to it, unless you set it up that way. But you know, the, it's, the pet food is, as far as the dog is concerned, it's free food. It doesn't have to go hunt it, it's just being delivered. And you know, one can imagine a scenario where there's sort of this bot economy and that's where the real hard work is done is by the bots. And the bots are interacting with this and there's lots of effort being spent by the bots. And then in the end, sort of the ultimate things that are relevant to us humans are just this tiny little froth on top of all this plot activity and that becomes essentially free. Just a thought I think that the way to think about that in, well, one way I've thought about that in the past and it was a little bit different from that, is it is true in the world today that there are natural resources, but where there's a mining issue, somebody might have a tantalum mine somewhere and that's very valuable because they have the tantalum mine, the other guys don't have the tantalum mine. And the thing that's interesting about the computational universe is we all have it and it is mineable anywhere, anytime. And it's just out there as a formal thing. It's mining the ruliad, so to speak. And that's a thing that sort of we can expect to be done anywhere. And I think. That's why in some sense it might make sense for sort of everything to become free eventually. Because in a sense we have this inexhaustible source of mining resources that is the ruliad, so to speak, and, or you know, is the pieces of the computational universe. Anyway, just a, just a thought there. Zayden is commenting. Ideas alone don't govern how science evolves. It's a combination of factors including technology, motor production of society and so on. I completely agree. I mean, my own life has consisted in this kind of sort of spiral of upward spiral, I'd like to think, rather than downward spiral, of creating science that allows one to create technology, that allows one to create more science and so on. And yes, there is a tremendously tight connection between the tools that exist and that are technologically created and the way that, that allows new ideas to be created. Then those ideas allow new technology and tools to be created. This has been both the history of my own trajectory and the history of kind of the, the history of ideas in much more larger scale. Memes is commenting. The sun's computation helps sustain us. Yes, let's see. Parker is commenting. They like thinking about machine learning as a black box that gets to human comprehendable product, but the reasoning that enables to get so that output is not understood. Once we understand what's going on in machine learning, Parker says we can be confident that it's output is sound. Yeah, well, the problem is we're not going to understand it. The problem is there's a trade off. If we want to lasso it back into the understandable, we're also preventing it from kind of galloping at full speed. That is the nature of computational irreducibility. If you're doing a computation that is irreducible, then you can't Reduce it. But in order to understand it, you have to reduce it to this thing that sort of fits in your mind and becomes a narrative. So it's kind of a, you know, there's two opposite forces. One is let's go irreducible and really let computation achieve everything it can. And let's go understandable where we're pulling it back into what humans can, can wrap their brains around. And that's more limited. I mean, as we get sort of, and, and the way we get to extend what our brains can do is through abstraction and the development of paradigms and the development of ideas. Because that, that's how we get to. Just like we're talking about sort of this analogical reasoning type thing of, of the neural nets, it's kind of like that's how we get to the next level up, so to speak, is we make this abstraction that allows us to not have to go down in the details of all of that irreducible computation. We get to take a piece of what we're talking about and make this abstraction of it that is good enough for what we care about. Yes, we don't know the precise details of how those bits move around, but our abstraction is sufficient to answer the question we want to answer. And I suppose that's the issue with this sort of meta model that I'm talking about for these neural nets is kind of, that meta model is telling one the parts of this computational process that are actually of interest to us as opposed to all the bit twiddling that's going on that ultimately isn't something that we can kind of attach to in a narrative way. So I think it's the case that as soon as the box is not black, the value of the box is decreased, so to speak. Let's see. I started playing chess lately and I noticed that high level machine chess is a lot like proof of computational work and willingness to commit it. Do you have thoughts on that? I don't really have great intuition about playing chess and things like that. I think that the, I mean, when we think about games, it's all a story of sort of game graphs and you know, how do we prune out the right game graph, a right path in the game graph. And that's really the same story as what we're doing in automated theorem proving. For instance, how do we find that path that gets us to the theorem? How do we find that path through the game that gets us to the winning state? Those are very much the same kind of idea. Piece of trivia. The person who first sort of talked about game graphs, I believe, was Ernst Zermelo in 1912, who was the person who was very much involved in set theory and who was also the person who was involved in the reversibility objections to the second law of thermodynamics back in the 1880s, 1890s perhaps. And so it's sort of interesting that that same thread carried through from kind of thinking about thermodynamics, thinking about those kinds of things, to thinking about set theory, to thinking about game graphs and so on. Yeah. So I don't. Let's see, Spare parts is asking, I wonder how much power one would need in order to run a mechanical computer comparable to a modern cpu. Interesting question. I know Babbage's analytical engine was going to be a big, big thing, you know, bigger than a locomotive. And it was steam powered. It would have been steam powered and it was able to compute. What was it? I think it could do seven figure arithmetic and it could do. Oh, gosh, I worked this out at some point. It was, you know, an instruction a second or something like this. So, you know, a billion times slower than a modern. Well, maybe its instructions were a bit bigger, but, but so give it. Yeah, I'm roughly a billion times slower than a modern CPU now to be able to do computations comparable to a modern CPU with kind of the steampunk type technology. It's. Or Victorian style technology. It's an interesting question. I mean, my guess is that, you know, Babbage, I think, had already imagined this kind of very, you know, he'd imagined cloud computing, so to speak, you know, if you want to back project from today's world. He imagined cloud computing and he imagined that you would have kind of these, these engines, these stations where you would do, you know, big computations and you would build a few of them. And a life insurance company would come in and compute its, its, its insurance tables and the navigation. People would come in and compute their thing. It's very much like the supercomputer story or the cloud story today. And you know, there was a branch of history where Ada Lovelace would have been the CEO of that operation. And we would have had the first soft back in the 1840s, you know, CEO'd by Ada Lovelace with Charles Babbage as CTO, so to speak. And that almost happened, but it didn't quite for reasons of, you know, Ada Lovelace got cancer. And you know, Charles Babbage never really internalized that vision properly and never convinced the British government To fund the thing in a more serious way. And you know, it didn't happen, that history never happened, but it was fairly close to happening, I think. Let's see, Spare Parts is asking do I think that modern AI systems are unique in terms of replacing human work or just another step in automation? I think they're just another step in automation. And you know, I was thinking today with, with you know, the chat GPT stuff and so on, you know, which can now write a very high quality essay that is, can be complete nonsense. I was asking it something about write a persuasive essay that wolves are the bluest of all animals. And it, it started talking about the blue wolf, Perenn canis something or other that lives in the Tibetan plateau and that its, its color, through structural coloration that is the same thing that butterflies use and so on. It's complete nonsense. I mean it really had me a little bit fooled there. It was like, really, is there a blue wolf? I, you know, I never heard of a blue wolf. Are there wolves in the Tibetan plateau? I don't know. You know, and it was, it was very convincing and it is a, you know, it's an interesting question. What, what happens to the world now that it's possible to write absolutely, you know, on, on their face convincing kinds of arguments and essays and no doubt marketing pitches and all this kind of thing completely automatically what I was thinking, what is the thing that will really change? So back when I was a kid, one's handwriting mattered. You know, you would get, you'd write an essay and you know, it's like part of the, the grade if one cared about that was, you know, could anybody read one's handwriting? And, and then handwriting just sort of went away for most purposes. You know, people type everything. And then there was a time when you could kind of tell a little bit about people by their spelling. You know, it's like, oh, this is a well educated person. They know how to spell and, or at least that, you know, if they, if they are dyslexic or something, they're using a dictionary, they're going to the trouble of using a dictionary, whatever it is, okay? And then that went away because there are spell checkers. And now if I get something where words are misspelled, it's like just this person was kind of, kind of dumb to do this because why didn't they switch on the spell checker? And so one of the things that I think will be a consequence now, for example is there won't be any excuse to write Broken English. Even if you started off with broken English, you'll be able to run it through, you know, through one of these systems and it will smooth the text, so it will sound just fine. Just as it can smooth the text with language translation, it can smooth the text there. So that's an example of automation. But, you know, if we look at the bigger picture of what jobs get automated, what do not. I did a bit of a study, which I never wrote up, unfortunately. I talked about it once or twice, but it was actually a bit frustrating because I looked at the data from the Bureau of Labor Statistics and its predecessors all the way back to like the late 1700s in the US of. Oh, it's a Census Bureau data, actually. Census Bureau data, sorry, asking what were the occupations in the US and you know, back in the beginning of the 1800s, the vast majority were agricultural occupations. And what I tried to do was to bucket occupations because the actual details of the occupations have changed over time. For example, I don't know what was the term? Like a chainman or something is not an occupation today. It's somebody, I think, who helped surveyors with their activities. Or a tinker is not really an occupation today, or a cobbler, you know, making. And those were occupations that were listed in the list of occupations. I tried to bucket these things to get sort of what the overall flow of occupations had been. And, and you know, what you observed was some things, you know, the, the general decrease of agriculture, the increase of, of everything to do with government, municipal services of various kinds, the increase of health care, the increase of education as occupational areas, the sort of military went a bit up and down, but not so much manufacturing, actually not as much decrease as one might have thought in terms of people. Just a lot of detailed things. And I was trying to conclude from this was probably nearly 10 years ago now. I was trying to conclude what can we learn about kind of the ebb and flow of human activities? And, you know, when automation comes in, what does it really do? And, you know, how does all those clerks who were, you know, doing, adding up numbers and, and keeping, you know, filing clerks, for example, you know, the number of filing clerks has dramatically decreased because now we're using online databases for that. And, you know, that, that, that thing got automated away. But what has continually happened is that, you know, new things open up, new occupations open up. And who knew that you could be a professional video gamer? Who knew that you could be a, you know, a linguistic curator? Who knew that you could be an AI psychologist? You know, these are all new. You know, who knew that you could be a prompt engineer for, you know, for figuring out, you know, how to create the prompts for, you know, some. Some creativity system of some AI system and so on. So there are these new things. Or who knew you could be a. An online community moderator? These are all professions that grew up as a result of technology, but it's something that, you know, hadn't existed before now. If you look at the places where humans are still very much in the loop, there's a lot of what one might call persuasion professions where humans are in the loop. Whether that's sales, teaching, I don't know, psychiatry, I don't know. You know, places where the human customer, in a sense, needs to be persuaded by there being a human there. And, you know, partly that's sort of the commitment of, oh, you know, somebody. A person is flying this plane. It's not, you know, there's commitment on the part of the person who's flying the plane that they're going to try and bring it into a safe landing, so to speak, rather than just, oh, there's just a machine doing it. And, you know, we're relying on the legal system that somebody has liability, and if you crash the plane and kill all the people on it, it's, you know, there are these places where the human in the loop is necessary for sort of the commitment of there really is a human who's putting effort into this. And, you know, you see this in business quite often where there's a kind of, you know, well, are you actually going to do the meeting? Take the meeting? You know, is. Is this level of poobah in this organization going to take the meeting? You know, that level, the commitment is a large part of kind of what's going on. And it's kind of necessary to have that slice of kind of human commitment. That's the story. And if you say, oh, let's just put a machine in place there, the human commitment is removed, the human persuasive component is removed, and that profession just doesn't work. So I think there are professions like that where human in the loop. If your customer is human, you're kind of. That's that's the thing. Now, there are other places where there are, you know, and I suppose the same is true in, you know, in. In a. There are settings. You know, we see this with things like, you know, music might be free, but the, you know, the concerts that are these experiential events that involve sort of human commitment of people being There and so on are, you know, part of the big part of the commercial story. You know, I think this notion of sort of humans are there just for the sake of humans being there is a thing that isn't going to get automated away just as the kind of high level, so what do we want to do isn't going to get automated away. Although there is an argument that leadership is always a small fraction of total activity. And if all you were left with was kind of everybody disappeared except the upper management, and it was just upper management kind of telling AIs what to do all the way down, that it is the case that a huge number of occupations are not the, you know, the top leadership, so to speak, they're making everything work down below that, so to speak. And if you could automate that away, there will be many fewer slots for those kinds of things, you know, so I, I think it's. Yeah, so, so, you know, it's, it's a really interesting question, what sort of, where do the humans fit in? I mean, there was probably a time when people said, look, as soon as there are bulldozers, it's all over for humans. Don't you know that having that whole team of humans to pick up the tree trunk and move it from here to there, that's what we humans are able to do. I mean, if we were building the pyramids today, I don't know how many people, millions of people involved in building the pyramids. You know, if the pyramids were being built today, the crew that built the pyramids would be incredibly much smaller than the crew that actually built the pyramids back in the day. And, you know, because there is automation, there are, you know, big machines and so on. And I think, you know, people might have thought, thinking about that future, oh my gosh, it's all over for the humans. You know, it's just going to be machines all the way down. But yet there's still, you know, 8 billion humans or whatever who mostly feel like they're doing something that is necessary for them to do. They can't just sit back, relax and let the AIs do everything that needs to be done that's not happened. And my guess is that it's going to continue to be the case that there's a sort of evolution of these niches that humans, for whatever reason, including the kind of sort of persuasion, commitment professions, you know, end up needing a human to be in those loops. Aaron is commenting that he may change his email signature to written by ChatGPT. Please excuse any nonsense. That's an amusing idea. You know, I long ago put something on, you know, for email that I send from voice to text systems that basically says dictated whatever, you know, to avoid the fact that sometimes the words will just be bonkers there. But, you know, it is an interesting question. You know, I get a lot of mail and from, you know, I make many people think I'm crazy to do this, but I make a, a real effort to try and do something when I, whenever I have anything sensible to say with, with mail that I get or my team gets. And you know, the thing I realized is, oh my gosh, there's stuff I get that, you know, could I tell you immediately that it was, you know, an LLM, you know, large language model piece of mail or was it real mail? I couldn't tell. You know, it's the same thing as when, you know, I have heuristics for telling am I getting phished? And the, and those heuristics, I guess, you know, I don't know how well those heuristics are going to work on sort of certain kinds of mail that one gets that can be generated now automatically. And it's, it's, it's interesting. I mean it's an interesting dynamic of what will be the, the, you know, probably phishing has caused ph. For. Phishing has caused people to read their mail a little bit more carefully. Maybe this will do the same kind of thing, I'm not sure. Let's see. Huh. Let's see. Crypto is commenting, it's temp. It's tempting to think the general AI will have a sort of digital version of evolution. Look, there's digital evolution all over the place. I mean there's evolution in products, there's the, you know, the automation of evolution is certainly. There are many places where that's been done and it, where it has. I mean, you know, a lot of, even the training of chat GPT I think had some sort of natural selection that was being done by actual humans saying that's a good story, that's a bad story and so on. And I think, you know, this is a digital evolution is well underway and how autonomous it will be, you know, the invisible hand, so to speak, you know, it can be and how visible the hand will be is not clear. What the best way to do that is, is not clear. But there's definitely some sort of evolutionary process on the way. And talking of which, I need to go and do some other things in a moment, but there was a question here from Ollie asking historically, how has written record keeping evolved? Will we ever revert to oral records and so on? Well, you know, written records were a thing of the creation of cities in the Babylonian times. That's when people started wanting to record things in a sort of more permanent way. That's when I suppose you were dealing with, you know, before that it was like, oh, yeah, I know so and so and so and so. And I remember they, you know, gave me a pot of, you know, a pot of honey or something and I owe them or whatever, and we can just remember that. And it's a small, kind of small group of, I know, 50 people or something, and everybody sort of remembers what was happening. And then by the time there were cities, much larger aggregations of people became more important to have records which were sort of more plug compatible between different groups and so on. And that's kind of how the whole process of writing and recording things and so on kind of got started and the process of having sort of how much one records. I mean, I myself have been a personal analytics nut, so to speak, and I probably recorded more stuff about myself over the last 30 years or so than pretty much anybody else has. And, you know, down to what keystrokes I type and all the emails I send and all the, all the things that I, you know, the documents I produce in every intermediate stage and so on and so on and so on. It's. One is recording all that stuff. Exactly. What one does with it is unclear. And, you know, it's rare that I pull it out and, and do things with it. But I think I did realize one thing, which is by the time, okay, so, you know, shocking fact of the day, if you feed typical, you know, college admission type prompts into ChatGPT and ask it to write essays, it does a reasonable job. It does a shockingly good job actually, including just making up stories that are just complete nonsense about how it tried out for a football team and things like this. And that was in response to some prompt about, tell us about a place where you failed and learned from your failure or some such other thing, and it makes up these stories and they're just absolute, total nonsense, but they seem very convincing. And if you ask it to do things like, oh, I asked it to write some thing about, write a sad story about this or that, and it wrote a very, you know, quite a convincing, quite a, you know, a, a sort of structurally kind of convincing version of that. And let's see, where was I going with this? I was talking about, yeah, the the written records and so on. So I realized that, You know, a lot of when people are trying to, you know, hire people do, you know, pull people into education type things. I've always believed that you really kind of have to do interviews if you actually want to find out, you know, who makes sense and who doesn't for some particular. In some particular situation. But an awful lot of people think, oh, you don't have to do that. You just get people to write essays and so on. I think the just get people to write essays thing just got exploded. And the kind of interactive kind of interview thing is likely to be the more, you know, the more convincing thing. And, you know, perhaps even the things I try to do with these, with these live streams where I'm just, you know, taking questions I don't know, are coming and trying to kind of sit and talk about them. You know, that may be one of the things that one sort of retreats to, away from the kind of the, the kind of just write an essay because it's like an essay can be generated automatically. But if you really want to check there's a human in the loop, you know, you ask them a question, you have them respond in real time. Now, in time there'll be, you know, video avatars of humans and things like this, which will get better and better and pretty soon, you know, and I, I certainly have every intention of training a bot on myself, so to speak. And, you know, I've got enough. One of the advantages of having recorded so much about my life is I have a huge amount of bot training data and that, you know, so it may be quite shocking that at some point I'll be able to produce a bot that can answer questions like these in a way that is a first approximation to the way I would do it. I'd like to think that there's sort of bigger arcs of putting things together that, that will not be achievable by something sort of pretty much other than me. But that becomes kind of the, the human signature then is, do you have all the memories, so to speak, and all of the sort of connections that can be made. Yeah, well, Sergio Comments that GPT4 and GPT5 are going to be amazing. Undoubtedly they will. You know, that's the question of what does that technology curve really look like? Because, you know, I've seen in my life a huge number of technology curves that have a rapid growth period and then they level off and eventually everything, all of these technology curves level off. I mean, you know, steam engines were a Big thing in the, you know, whatever, beginning of the 1800s and so on. And they had a very rapid growth rate. And, you know, every month somebody was putting out a new kind of steam engine. And it was, it was big. But then, you know, steam engines plateaued. And the question is, and a lot of kinds of technology, you know, they achieve a certain thing, they end up being spectacular for that thing. People find use cases for that thing. And then everybody says that's amazing. And that's really the issue. You get a certain thing, you get this blob of technology, and then you ask yourself, it achieves these things, what are its use cases? And then can that blob itself expand to be, to do more things, or is it more a story of finding use cases? And it's unclear to me. I mean, I know that networks that do classification, things like that for image processing, things of this kind, you know, we did a bunch of things with that back in 2016, 2017, things like that. Maybe it was even earlier than that, maybe it was 2014 even, I'm not sure. And yeah, you could get a certain distance and that, that allowed one to do a bunch of things. And they're pretty cool and they have interesting use cases. But that in and of itself is, isn't like the thing that's driving what happens next. So I, I don't really know at what point, you know, you know, did we just reach the threshold? So, for example, with language translation, we got there pretty much. Pretty much language translation is now doable, human language translation. And you know, there will be improvements. But that hurdle got, got crossed, as did a bunch of other kinds of things. So it's unclear to me exactly what the growth curve looks like. And one shouldn't make the assumption that just because it's growing rapidly now means it will always grow rapidly. I mean, that's the famous mistake in market, in financial trading and so on. Where you say it's going up now, it's going to keep on going up forever. Look, it's been going up for the last two months. That means it's going to go up forever. Well, no, it doesn't. But predicting when the end is, is very hard. And, you know, where's the peak? Okay, that's a good question. Prototype asks while we're talking about interviews and so on, Will the interviewer care if the candidate is an AI? That's a really good question. If you do an interview and the interviewee is just amazing at being able to respond for the things they need to do, to do the job. And it turns out the interviewee is an AI. Yeah, it's a good point. You might say, let's just hire them. Now. That creates this very interesting dynamic of let's hire the AI. What if the AI was a bot? It becomes very weird. I mean, the labor market then becomes very weird because the bot might be free and the bot might have no economic. It all gets very strange. But, yes, that's a cute point. OG asks, has Chat GPT passed the Turing Test? Can it pass the test soon? I think it has effectively passed the Turing test. I mean, I'm sure there are details where you can tangle it up and you can do this and that and the other, but I think for most people, you know, as a practical matter, yeah, we pretty much got there for that particular version, you know, if you say did it. So. So now we. We have. Here's an interesting version of the Turing Test. Could it pass for Alan Turing? Interesting question. I don't know. We don't have a single recording of Turing, but we know, you know, would it. The. That's a question. Would it pass for your average middle school student? Yes, is my view. Would it pass for, you know, the. The kind of. The finest humans we have at making sort of big thematic conclusions? No, it will not. Yeah, but, yeah, it's an. It's an interesting moment, I suppose. Prototype is saying, suspect the major deployment of AI in short term will be phishing. That's an interesting point. For the time being, it can't replace regular employees, legitimate businesses, because it can't be held legally. Response culpable for. Because it's not conscious. Oh, that's such a philosophical mess. But for scammers, that's not impediment. Good point, good point. I mean, this is the same thing. This is the story of all these use cases. And a lot of use cases for machine learning. Machine Learning is like 80% successful, 20% disastrous. But there are use cases where 80% success is enough. Classic examples are feeds for social media or search Engine results. If 8 out of 10 results on the first page are what you want, everybody's a winner. Two are completely nuts. You don't care. But on the other hand, there are other use cases where if 20% of it is bonkers, it's disastrous. So, you know, there are these different use cases that you have to think about. And that's, you know, that's kind of the. The way that, that, that works. Now in terms of legal responsibility, you know, I think that's a very Complicated issue because it is clear that there are automated systems that do things which kind of extend the reach of the human. You know, you could say when you crash your car, you could say, I didn't do it, it was the cruise control that did it or whatever, or the self driving feature that did it. But I think, you know, the theory of liability tends to be the case that, you know, it's looking back to was there a human in the chain, so to speak. And I suppose, yeah, I mean, it's an interesting question what, you know, whether the human in the chain ends up eventually being the builder of the AI system. Fortunately, that didn't really happen with software. It could have been the case that, you know, the software industry was kind of killed by people saying, oh, you know, it was the software that made me do that. And then the software maker would be liable for all the things that happened as a result of somebody doing that. So, you know, and that, that didn't happen in that, in that domain. And yeah, it's a. Well, okay, there's a different topic for another time perhaps about sort of the, the ethics of AIs and responsibility and so on and, and the notion of consciousness and what, what it takes to have, for a thing to have sort of a notion of responsibility and so on. All right, well, I should wrap up there, but thanks for a lot of interesting questions and comments and you know, as you probably on these topics that are really what they call it, you know, an unfolding story, so to speak. I mean, I've been actively thinking about some of these questions about sort of the, the modern state of AI and so on. And so as I talk about this, you know, in successive live streams and weeks and so on, no doubt my position will change somewhat as I understand more. But I, I've appreciated even today, I think I've probably from the beginning of this discussion to the end, I made a little bit of progress in understanding, you know, how I, how I at least think about what's going on in these areas. And it's something that I think there's a few more kind of cognitive jumps to be made. And yes, I have had some fun sort of asking ChatGPT about some of those things and it's interesting to see the extent to which it's like, oh yeah, okay, that's a thing worth thinking about. Worth it kind of jogs me into thinking about that particular kind of kind of thing. Anyway, well, thanks for joining me here and see you another time. I'm not quite sure which day I'M doing one of these again. I, I'm just a, I, I'm kind of like one of these humans who is like the humans where the AIs are in charge. I'm just, I just look at my calendar and follow what it says to do because some, some overall system was built in the past that causes it to have that structure, and I'm just following through on it. All right, well, thanks for joining me and see you another time. Bye for now.