History of Science and Technology Q&A (August 2, 2023)

Hello everyone. Welcome to another episode of Q and A about history of science and technology. And let me see, we have all kinds of questions here. IMM asks, do we know what the first piece of technology was? Well, what is technology? Technology is somehow taking things that exist in the world and setting them up so, so that they can be applied for human purposes. So the real question is how did people sort of, there were just the pure humans, like you know, the naked animals, so to speak, running around. And then there are things humans made use of from the outside world to do things like for example, they might have made, you know, flint axes, they might have made pieces of clothing, they might have made things like that. I would think that probably the first thing one might reasonably think of as technology was probably things like axes where you're taking a stone, like a flint, for example, and you're napping, just hitting the flint with another flint to kind of make it so that eventually, by the nature of that kind of stone, eventually you can get these sharp edges which you can then use as a way to go, you know, attack critters or whatever else one wants to do. So I think that's probably the sort of first example. But now this question of was something a piece of technology or not? People might argue about this in great, at great length because like you find rocks that arrange in a particular way in some, you know, Paleolithic archaeological site. Was that a technologically set up thing? Was that for a purpose? Or did those rocks just happen to be thrown in those places? Likewise, it's kind of like these cave pictures where, you know, there are lots of, well, at least there's some caves where there are just a bunch of hand prints. Are those handprints just randomly put there? Just because somebody was like, you know, I don't know, having a party and thought it would be amusing to just put their, you know, know, put their hand, dip their hand in paint and stick it on the wall. Or was there some great significance to the arrangement of those hand prints. That sort of a question of was that technology or was it not? Was that something that was set up for a purpose or not? And I think the same when one looks at things that various kinds of non human animals do, you can ask the question, you know, is that technology? Is the beaver dam a piece of technology? It's at least potentially beaver technology, even if it's not human technology. Or then you could ask, well, you know, is the, is the silk that's spun by a silkworm? Well, that's perhaps technology for the silkworm. But it also turns out to be technology for humans because humans can make use of that silk as well. So and, or, you know, more extremely, you know, the trunk of a tree is presumably something where, you know, the tree is assembling that for the purpose of getting its leaves up high and all that kind of thing. But also, we humans can make use of that. So I think it's a. It's a complicated question, but I would say that the, of the things we know about some of the earliest kinds of things will be things like, you know, stone axes and so on. That's, that's probably very early examples of taking something from the world at large and kind of customizing it for human purposes. Let me see, I'm a little bit. I see a whole bunch of interesting questions here. There's one from Aaron. I just noticed Alan Turing had not died at the age of 41. Let's see, was he 41? He was born in 1912 and he died in 1954. So give or take, yeah, that kind of age. If Alan Turing had not died in his early 40s, what might he have worked on during the remainder of his life? You know, one thing about Alan Turing is he was a, a sequential projects guy. He worked on something, he worked on, you know, he finished it and then he kind of moved on to a different thing. He was not. There's some people where the arc of their careers involves kind of. There's the big thing you're building and it's kind of. And that's going to span decades of life. And then there are other cases where it's more like you do a thing and then you see another interesting problem, you go work on that, that takes a few years. Then you go see another interesting problem. You work on that. Alan Turing was more of that second kind rather than of the first kind of the giant arc of activities. So, you know, in his life he had worked on things like the Central Limit Theorem. That was one of his early projects of proving the Central Limit Theorem. That's something that John von Neumann, when he wrote a recommendation letter for Turing, that's the thing he mentioned in the recommendation letter, not the universal Turing machine paper. And I don't think Alan Turing thought that his paper about computation, universality, I mean its Official Title in 1936, Paper on Computable numbers with an application to the enshridence problem. The Enshidance problem was the decision problem of Hilbert. That is the question of if you've got some. Can you determine by mechanical Means whether some specified statement, some specified logical statement is true or not. Like, for example, a statement might be. This particular equation involving integers has no solutions. Is there a mechanical procedure for deciding if that's true? Well, I think when Arnold Turing wrote his paper that led to the idea of Turing machines and universal computation, he didn't think it was that big a deal. Also because he had this model of computing that was based on the operation that bank clerks have. He was kind of idealizing what bank clerks do. You kind of take something from old bank clerks before there were computers and so on. You take something from one ledger and you go look at and you put it on another ledger and you do this and do that. He kind of imitated and idealized that in the tape for a Turing machine. It's kind of interesting to look at that because I'm about to be working on observer theory, which is kind of a way of idealizing the operation of an observer, much as Alan Turing tried to idealize the operation of a computer. And by computer, I mean a person who did computation, which was the only example of a thing that did computation back in those days. But anyway, I think it was not clear he didn't think of that paper as having the kind of dramatic importance that it's turned out to have. But in the arc of Turing's life, he then ended up working on cryptanalysis during World War II, on kind of the things that led to information theory, I think. And then he worked on other stuff. He worked on the practical programming of the Mark 2 computer in Manchester. No, the. The Ace in Manchester. And early electronic computer. And so we have, you know, computer documentation as written by Alan Turing. And we have coding as done by Alan Turing. He was a pretty hacky coder. And then he got interested in theoretical biology and he was interested in the phyllotaxis, the, you know, how leaves grow out of the shoot of a. Of a growing plant. And, you know, he had this thing where possibly this thing that one often hears from applied mathematicians and so on. Consider a spherical cow, I think might come from Alan Turing, who was interested in the dappling patterns on cows and how they would be generated from reaction diffusion equations. But he, as an approximation, considered the cow to be spherical. But it's very notable when he worked on theoretical biology in the early 1950s, he didn't even imagine that one might use Turing machines as models for that. He just went straight to partial differential equations from kind of standard mathematical physics. And then he had other things he'd worked on chess playing machines with his. Friend Champanone. What was Champanone's first name? I can't remember. David Champanone. Maybe. Champanone was a friend of Turing's as a student. Champanone himself later became fairly well known economist. But he was a person with whom Alan Turing worked on chess playing machines. And Alan Turing worked on having a mechanical device to compute the zeta function and managed to compute further in the rim zeta function than people had been able to do before. And in 1950, famously, he wrote his paper about AI that was informed by his Turing machine work, but was more by that point informed by the fact that there were starting to be actual electronic computers which were in a sense informed by Turing machines, but by a rather circuitous route of von Neumann having been interested in neural networks and McCulloch and Pitt's work in 1943. McCulloch and Pitts relied on Turing's work in 1936 as their evidence that in McCulloch, Pitt's neural net could really do universal computation. And they then sort of assumed that universal computation was kind of the big story of computing things. That would be what brains would be able to do. But I think by the time Alan Turing was working on sort of AI and the Turing test and so on, he was more thinking of practical electronic computers of the kind that he was by then using. So what would he have done next? He was interested in physics. He didn't know that much about physics, but he was interested in physics and he was interested in kind of foundational questions. And I wouldn't be that surprised if he wouldn't have worked on kind of trying to find sort of more fundamental ways to think about physics of the kind that I've spent a bunch of effort on. I think, you know, Alan Turing never simulated a Turing machine. Maybe he would have done that. If he'd done that, he probably would have discovered a bunch of things that I discovered in the 1980s about what simple programs do. He was more, you know, in his construction of his universal turing machine in 1936, he was much more of a can we, as engineers, so to speak, make a machine that has all of these sort of capabilities of doing universal computation? He was not given the concept of a Turing machine. Hey, let's just start enumerating Turing machines. He knew you could enumerate Turing machines. That was actually important to his argument. The fact that you could specify a Turing machine as ultimately an integer was something that was important to him because he had to be able to give the specification of A Turing machine as data to another Turing machine. So he had to know that it was possible to give the specification of a Turing machine as a number and therefore as something which could be enumerated. In fact, that was also important to his argument about the decision problem and the undecidability of the decision problem and the diagonal argument that he gave for that. So he knew that Turing machines were enumerable as numbers. But I don't think he ever imagined the idea you should actually look at Turing machines. 1, 2, 3, 4, 5, up to a million or something. He didn't think of that. He certainly had the technical capability by the end of his life to be able to use a computer to look at those things, but he never looked at them. Those things were looked at around 1960 actually by people like Marvin Minsky who enumerated the first 4096 Turing machines. They're just the ones with two states and two colors and found that they didn't do anything interesting. He has a quote in his book about finite and infinite machines that he published in the mid-1960s that says about very simple Turing machines, that the kind of analysis of these 4096 Turing machines is laborious and best carried out with a computer, but that it is unpublished and unpublishable on the grounds that it's just too monotonous and sort of automated to be interesting. But anyway, Alan Turing never did that. You know, my own guess would be that Alan Turing would have been, would have gotten into the, into sort of computer entrepreneurism to some non trivial extent. I think he was a person, you know, he was sort of an academic, but he bounced around between kind of working at sort of the emerging computer center in Manchester, kind of being a fellow of this Cambridge college, kind of being involved with the National Physical Laboratory in London. And in the war he was involved with Bletchley park, which was after all a military research organization. I think he was not a sort of dyed in the wool kind of I'm going to be a professor all my life kind of person. I think he was more a project by project kind of person. And I think he would have been very likely drawn into kind of early computer entrepreneurism. And I think as one could see that sort of happening in his work on the Manchester computer. I think he was frustrated by working on projects where other people were in charge. I'm not sure how much he would have been. I think his immediate self image would not have been that he would have been able to be in charge of something Although my guess is he would have been successful at that. I think that he would possibly have gotten involved in, you know, more kind of sort of public outreach kinds of things, particularly if he'd kind of lived through the 1980s and so on. I mean, he was not well known until probably the 1990s actually. And you know, he was a person where people had heard of Turing machines, but that was kind of a geeky thing in theoretical computer science. As Bletchley park and the activities During World War II got declassified, he became a bit more heroic in that regard then kind of had a boost with Andrew Hodges biography and kind of in the end being sort of an icon of gay rights for a while, that sort of, I think boosted his general public visibility. And now some of that might be kind of, you know, he died by eating an apple poisoned with cyanide. And it's kind of a long debate of, you know, did he kill himself? Was he poisoned by somebody or was it an accident? I have to say my. I think the likelihood that he killed himself, I believe the likelihood of that is very low. I know a bunch of people who knew him. I've talked to them extensively about this. To quote one of them, if Alan had going to be. If Alan had intended to kill himself, he would have written a whole book about it. I think that's a very unlikely possibility. But you know, part of the myth has to do with the fact that, you know, there was a lot of kind of, I would say non gay rights kind of stuff that went on that we would now think of as totally crazy. And you know, but the truth from at least people I know who knew Alan Turing is that that stuff, he thought that stuff was kind of crazy, but it didn't really profoundly bother him. Certainly not to the point where he was going to kill himself over it. But that was part of his public visibility now has been associated with that essentially connection to that sort of martyr type aspect. So if you take that out of the picture and you imagine what would have happened with Turing if he had lived another 40 years or something, 50 years and you know, he probably wouldn't have. I don't know whether he. What level of public fame he would have achieved. Probably it would have depended on, well, probably a certain amount more for his script analysis work during the war. I think he was. And I think he would have been pulled a bit more into kind of the, the public outreach arena. Like his radio broadcast from 1950 about which led to the Turing Test idea. I think my guess is he would have worked on a series of. Well, there's an interesting problem. Can I work on this? There's an interesting problem. Can I work on it? I think some in physics, I think he might very well have wound up doing, starting a company in some form or another, at least being involved in a company that was starting. And I think he probably would have tried writing a bunch of software. Whether he would have done that successfully, I don't know. He was not a person who was. He was a person who was very clean thinking when it came to core ideas. He was not particularly clean thinking when it came to things like notation and presentation. I mean, interestingly, he did write a paper once, I don't think it was published, but it's in his collected works about mathematical notation, kind of musing on issues of kind of how that's set up. And I think, you know, one could imagine he was certainly interested in mathematics. One could imagine him trying to write kind of mathematically oriented software and so on. I don't think he would have been a great user level designer. I think he would have been good at understanding what the sort of the core point was, but not so much at making something which other people could readily understand. Although it has to be said that the exposition in his papers is quite good, although the notation is quite often quite obscure. I mean, in the original paper about universal computing, he has what he calls Greek letters and what he calls German letters, which are Gothic script kinds of things, and the whole thing gets very, very Gothic. Maybe in his handwriting it wouldn't have looked so bad, but by the time it's typeset, the thing is pretty difficult looking. Anyway, that's a thought on that. Let's see, you know, the question. I think I've written things about Alan Turing, particularly in connection with the centenary of his birth in 2012, and I have to say that there are an awful lot of things that Alan Turing was interested in that I've been interested in too. And there's a certain sense in which, well, if Alan Turing had lived longer, he would have done all the things that I've had the fun of doing before I did them. And so then he would have built a platform on which people like me would have been able to operate and do different kinds of things, probably. But there's a certain argument that a bunch of things. I mean, I've been sort of amazed when I found Alan Turing's paper about mathematical notation. It's like, oh, my gosh, I've written about that too. They're just an awful lot of things. And he wrote about things like could you take the contents of Encyclopedia Britannica and encode it for a computer? And how difficult would that be? And he had an estimate of how difficult that would be. In a sense, that thing is like WolframAlpha or these days, like an LLM. Actually, it'd be interesting to look back at his estimate. I did look at his estimate relative to the effort that it took to build WolframAlpha, and he was overly pessimistic in how difficult it would be to make sort of extract the computable data from the world. It's an interesting question relative to the training for an LLM. You know, he didn't think about at that time, you know, it's a loop that didn't close. Actually. From Alan Turing came McCulloch and Pitts and neural nets. I don't believe that Alan Turing closed that loop and thought about neural nets. I don't think that loop ever closed. I think he was really thinking about AI in the context of more kind of symbolic kinds of computation and things a bit more like Turing machines. But in any case, I have been a bit shocked at the extent to which things that Alan Turing was interested in and wrote about are things that I've ended up also being interested in. I know that. Well, in just a whole collection of cases or like combinators, something I was interested in. I find out Alan Turing was interested in those too. So I don't know what one can deduce from that. But, you know, I think that the key point is that Alan Turing was interested in foundational things, and so have I been. And he was interested in kind of formalizing things in a certain kind of way, and so have I been. And that has led to some overlap in the kinds of interests because, well, there's perhaps, you know, you're concentrating on the foundations of fields like mathematics and so on, and there's kind of only. There's sort of one path down in those cases. Boy, Des is asking, what about if von Neumann had lived longer? Von Neumann died at the age of. Died in 1957 and he was in his 50s, I think, you know, von Neumann was a very quick off the mark again, not so much of a large projects person, more of a problem solving kind of person. And by the end of his life, he was doing an awful lot of work for the US military and an awful lot of work for computer companies, basically. And I would suspect that those are the directions that he would have more gone in than in Kind of diving into another sort of area of basic science. I think he was much more, you know, he was a commissioner of the Atomic Energy Commission and he was kind of a key consultant to IBM and he was sort of major figure in the whole military Cold War kind of story. And I think that's kind of a, a place. My guess is that's more the direction that he would have gone in than sort of diving into other areas, basic science. I'm, I'm sure, you know, at the time when he died. Well, okay, there was one area of basic science he was working on, which is this kind of computer in the brain theory of self reproducing, automata, these kinds of things. So, you know, he might have pushed neural nets more because he was interested in that and you know, whether he would have, you know, pushed them hard enough to get to deep learning and so on. I don't know. The computers didn't really quite exist then to be able to do that. But also the kind of the will to realize that high dimensional, deep learning, high dimensional neural network learning was going to work in the end even when sort of the low dimensional toy models weren't necessarily going to work. I don't know if would have realized that. I would bet against it. I think he was a person who was very good at problem solving with certain tools of mathematics and so on that he knew and kind of logical thinking and so on. But he had a habit of kind of missing the point on the kind of deeper issues of, you know, not really understanding universal computation until that kind of, you know, I mentioned he didn't really understand Alan Turing's original 1936 paper until years later. He didn't really understand, you know, he didn't, he was working on the foundations of math, but he didn't conceptualize the idea of Godel's theorem. You know, he had a formulation of quantum mechanics which I think in the long view of history will have been seen to have kind of elided a bunch of things about measurement that confused people a lot. But, but you know, but on the other hand, you know, you find his, his fingerprints in all kinds of fields. Like if you go to metallurgy and you look at things like, you know, what's the average number of, of, of or kind of a collection of soap bubbles, you know, what's the average number of sides to a soap bubble? Well, you get work from von Neumann in place like that. You find his kind of him touching an awful lot of areas and adding a certain kind of mathematical problem solving to those areas. And that's, I think, his primary mode of operation. Let's see. ABHI asks. Asks who was the first. Was the first time when statistics was used to predict something? You know, statistics, the word means comes from kind of features of the state by state. One means state, as in thing run by government or something. It's kind of like, what are the sort of aspects of the kind of aggregate characterization of the state of the state, so to speak. I think John Graunt is the person who is usually credited with having had the first kind of aggregation of that kind. He had a book in the 1600s called Bills of Mortality, rather grimly, which was a description during the plague, I would guess, 1665, maybe that plague in London. It was the grim list of how did people die and where did they die? Which boroughs in London were this number of people dying in? It was kind of an early kind of way of aggregating those kinds of things. Now, of course, even antiquity, there have been plenty of censuses of kind of, you know, count the people in this place or that place. But primarily the. The purpose of that counting was to, you know, levy taxes and to figure out, you know, how much tax should there be on this side or the other thing. But I think that was more of a. I don't think that. I mean, I think it was more like, okay, king, here's a bunch of tax, rather than okay, King, there's 11,312 people living in this part of the kingdom, et cetera, et cetera, et cetera. I think that that wasn't so much a thing in antiquity anyway. The thing that's usually attributed is the first kind of statistics of. Well, I suppose John Graunt also did things like making averages and so on, working out the mean, the average number of people who died in a given week or something, those kinds of things. But I don't think that that was a time when there were predictions going on. I think that was more just a. Here's how it is. I think there were things. Oh, gosh, by the 1800s and so on, there was lots of visualization going on. People like William Playfair has the. What's it called? This kind of atlas of statistical breviary. Maybe, I'm not sure, but it's kind of an atlas of features of the world, like the summary of the number of grain imports from this country to that country over the last 50 years, those kinds of things. But I don't think he ever predicted. I don't think he extrapolated from the data that was there. So I think that was more here's how it has been rather than. And from this picture we can now predict what will come next. I suppose that the early predictions of things, I mean even in Babylonian times there was kind of here's the position of this planet, we can predict where it will be next. But that wasn't really a statistical prediction. That was more something that was sort of purported to be a mathematical calculation of what would come next. You know, I'm wondering, I mean in other places where statistics arose, you know, when Gauss discovered the asteroid Ceres in 1800, I think there was all kinds of sort of telescope errors of kind of what you see and that's what led to the Gaussian distribution and so on. But that wasn't again really that wasn't a prediction of what will come next. That was more just a. If you make many measurements, this will be the distribution of errors in those measurements. I'm trying to think the. Certainly. Well then there were people like traders again. I think There was around 1900 the. Oh my gosh, what's the person's name? Because with L, I can't remember it, but a person in France who was analyzing the kind of random motion of stock prices on the Paris Stock Exchange about the same time as Brownian motion was being analyzed and kind of looking at those random wiggles. But I think that that was more a. Here are the statistics of those wiggles. Not so now we know that grain prices are going to go up. There was sort of a long tradition of so called technical trading. By the time there were those kinds of trading charts and so on, people sometimes they got called chartists would be people who would look at the charts and say it wiggled up, down, up, down, up. So it's going to keep going up. That was kind of a particular configuration. A lot of those configurations got names but that were kind of the, you know, really their random walks and some first, you know, very good approximation but yet you can kind of see just like the constellation of stars. You can see there's a, you know, there's an arrangement that looks like a duck there and you know, you can see it in the stars just as a human pattern matching thing. And so similarly in the somewhat random walk of stock prices you could see those kinds of things and lots of people had still do sort of take very seriously those kinds of things. And there's a lot of Fibonacci based kind of observations of kind of Fibonacci like sequences in the wiggles of stock prices and so on. And people make trades based on those. And those are cases where they are in some sense predicting maybe not so much from statistics as from sort of fitting some kind of model. Similarly with time series and so on, it's kind of like, can you extrapolate this time series and are you doing that by the statistics of what's happened or are you doing it by some kind of a very common time series prediction? There's an autoregressive model, like an autoregressive moving average or armor model, where you're looking at essentially making an equation for what the curve is doing. That is a deterministic equation that will predict what's coming next. More so perhaps than something where you are saying from statistics, this is the sort of, this is the distribution of what will come next. It's more, we've almost got a model for what's happening, let's just use it. I would say that, gosh, I'm thinking in, by the time we get to things like World War II and the operations research that was done at that time, again, I would say not so much statistical prediction, more that's, you know, use linear programming to figure out if you make this number of widgets in this place and how many widgets do you need for that and so on. You know, in econometrics, which developed, people like Morgenstern, von Neumann, people like that were involved, but there were, gosh, I don't actually know who were the kind of leading lights of that field. I think those two were definitely very involved in the kind of statistics applied to economic indicators. And now does one make predictions from those statistics? I certainly, in modern times people do that. I mean in, in, you know, in the end, the story of quant finance is a story of kind of using statistics to make predictions. And that's something that really didn't develop in serious form until the 1960s, 1970s, even 1980s of kind of statistical arbitrage and the whole kind of, well, that's again, it's kind of use statistics on existing data, take bets, see what consequences those have and so on. I mean, I'm trying to think, I'm wondering whether there were sort of quant traders perhaps in the technical trading chartist kind of crowd from much, much longer ago. But one doesn't necessarily know that because it's just like, well, so and so managed to make a bunch of money but they're not going to tell anybody how they did it type thing. But interesting question. I'm Kind of thinking, I mean there was certainly. Yeah, I think there may have been political arguments that were made on the basis of statistical results. And perhaps in the early part of the 20th century that might have been a thing. I mean certainly happens today, but I think that might have originated much longer ago there. Let's see. Oh gosh, so many interesting questions here. I see Michael here asking, commenting, they recently watched the Oppenheimer movie and seeing portrayed Einstein Godel in Oppenheimer, the small lake. I said I realize there are barely any relevant theoretical insights in the last decades, especially compared to 100 years ago. What does this mean for the science of the next hundred years? I dispute the claim that there are not great theoretical insights in the last few decades. I even think I've been involved in a bunch of them. But one won't know they were great theoretical insights for decades more. And it's only in retrospect that one could see, yeah, that was an amazingly great thing at the time. When Godel's theorem was developed in 1931, it was a thing. But how big it really might have been and the fact that it sort of backed into the idea of computation, nobody could see that at the time. The fact that General relativity from 1915 turned out to be a big thing in the sort of definitive theory of space time, at least until recently, again was not something that was known. It wasn't until the 1960s that really the serious significance of that became clear. So it's very hard to assess what will be significant until quite a long time afterwards. I think that in our current times I would like to claim that our physics project, for example, and the things around it and multi computation, the whole sort of developments of the computational paradigm, the exploration of the computational universe now, all these sort of discoveries about what LLMs can do and so on, these are big deals. These are certainly as big as what was out and about 100 years ago. I mean, I happen to think that there was a period of rapid growth in physics around the time of the developments of quantum Mechanics in the 1920s and so on. There was the development of computation in the 1930s. I think the kinds of things that we have now with our physics project, well make certainly the developments in physics, we kind of, that's sort of the direct follow on to what happened 100 years ago is what we are now doing. And it is a similar kind of step to the step that was taken 100 years ago. I also think when it comes to metamathematics, there's been a long Fallow period when not a lot of foundational things have been figured out. I think my own efforts and kind of the physicalization of metamathematics are another step. The kind of a direct follow on from what happened in the 1930s and so on in that field. I think that we're not there yet. But in biology, for example, there are a boatload more facts that we've accumulated in the last half century or so. There isn't yet theory. My guess is there will be and it will be big. And there really has never been a theory of biology. Well, there are really two theories in biology. Natural selection and the idea of sort of digital nature of DNA, the central dogma and so on. And that whole kind of assembly of ideas that center around the sort of digital nature of genetic information. But there will be big theories there too. And those are things which are I think emerging and will emerge from the kind of formalism that for example, has been exposed in our physics project. And there's this then, this separate branch, as I say, about things like LLMs and AIs and so on. I think what we're going to see in the physics and the science of the next hundred years is science sort of builds up higher and higher levels of abstraction and that's kind of how one gets to talk about bigger and bigger things. I think that this kind of idea of, well, the ideas of computation that's getting absorbed, not fully absorbed yet, phenomena like computational irreducibility, definitely not fully absorbed. Even though it's 40 years old now, the kind of the computational foundations of fundamental physics, that's really very new, you know, three years old or something, that's going to be something that gradually gets more and more absorbed. And what that means both in terms of our understanding of physics and our way of exporting kind of the developments of physics to, and the results of physics to other fields, to distributed computing, to biology, to linguistics, to economics, and so on. I think there's going to be sort of dramatic progress as a result of these really to me, these two kind of paradigms, the computational paradigm and the multicomputational paradigm. And I suspect that the computational paradigm kind of unlocks a bunch of kinds of modeling that we weren't able to do, that we now are. The multi computational paradigm is now at the point where it starts to be able to unlock a lot of questions about what really is the theory of natural selection, how does it really work? How do we make a theoretical framework that explains how species work and how the tree of life Works and so on in economics, how do we really make a theory that tells us things about how kind of the network of transactions in an economy leads to all the phenomenology it leads to and so on? And, you know, those have been places where we've sort of been blocked from having a theoretical approach. I suspect that the whole multicomputation idea will give one a theoretical approach to those things. And these will be a very fertile time. I mean, in the next probably decade or so is probably the time when the more progress will get made in the theoreticalization, so to speak, of a bunch of fields like biology and economics than have been made in many hundreds of years in the past. So I think we are far from it being a time when sort of nothing interesting is happening. This will be a time, as happened 100 years ago, when physics and metamathematics and, well, the emergence of computation and so on. That was a very fertile time. This will be another very fertile time for these kinds of theoretical sciences. I have to say I worked at the Institute for Advanced Study, which is where the pond you mentioned is located, and I worked there in the early 1980s. And the pond was not looking the same in the movie as it did when I knew it. In the 1980s. It was looking a lot more mangy. Now, I don't know whether that's because I haven't been to that pond since the 1980s. So I can't speak to what was happening with the pond today. I assume that the movie did not go back and recreate the pond of the 1930s. So I assume that's the pond as it is today, which is looking a bit mangy. But in any case, and there used to be a lot when in my time there were a bunch of ducks that hung out on that pond. I don't think I saw any ducks in the movie. But who. Who's to know? I think in, you know, it's interesting because by the. Well, you know, the 1930s was kind of the. The late side, the later side of the developments in things like metamathematics and quantum mechanics and so on. And we're not. We're still on the early side of the 21st century kind of burst of activity in the formalization of science, I think. Oh, my, the gosh, so many questions. And I'm skipping ahead to ones that are coming in here. Internet asks, where do you see applied psychology in a decade? Is the quantification of behavior and thought going to be a shift as advertised? You know, I think LLMs are the big break for things like applied psychology. I think that they behave in surprisingly human like ways. There are ways in which they're not very human, but if you kind of look at them, there are a lot of things about the way learning works, the way reasoning works, the way kind of we deal with common sense and so on that I think are well modeled by LLMs. We don't understand how LLMs work. There isn't yet an LLM psychology. There will be. There isn't yet an LLM science. I've been starting to pick away at that and hopefully other people will do this too. As one tries to understand what is the kind of narrative that one can tell about why does the LLM do this or that thing? Why is it the case that if you give the LLM too much kind of training data at the end and you don't kind of repeat things it learned before, why does it get confused and obsessed with the new data and so on? It's kind of like cramming for the test. It only remembers that or something. Why does that really happen? Why is it able to do kind of, you know, zero shot or few shot learning of kind of being able to be told, you know, act like a such and such, and it can suddenly be in character doing that. How does that work? And you know, how do these things work? Well, the LLM is kind of a lab where there are no, you know, there's no institutional review board at least yet that tells you, you know, the ethics of doing an experiment on an LLM. You're free to experiment with it and take it apart and see what it's doing. Now making a narrative science from that, that's not an easy thing to do. And it needs new ideas about how to kind of make formalizations in science. Maybe some of what we've learned from the physics, the whole multicomputation idea may be helpful for that, but I think it's still a very challenging thing. So I think that's kind of, in my opinion, that's the big break for psychology and neuroscience that one didn't really see coming. One now has a model, so to speak, that one can explore in much more detail. I would say that in terms of the can you predict what the humans will do? I mean, the other thing that's of course coming is, is much more data about what's actually going on in brains and much more detailed recording of neural activity and so on. And you know, we have 100 billion neurons in our brains and being able, you know, there Will come a time when by some feat of signal processing this, that and the other, when we're able to just have this giant readout of what are these 100 billion neurons actually doing at every moment. Now you say, well, that isn't that amazing. Then we've solved neuroscience and psychology. Well, not quite. I mean, we could get a display of every gate and, you know, hundreds of billions of gates in our computers, in the microprocessors of our computers. We'd say, what's happening with all of these gates at every moment in time? But still, understanding what's really going on from that microscopic data is very challenging. It's kind of like the ultimate example. You know, if you're trying to decompile something, reverse engineer something, you've got the machine code. But what did it come from? What were the functions that compiled into that machine code? What was the kind of human understanding of what was happening? What was the bigger narrative picture about what did this program do, rather than just what are all the bits that if run on the machine will work in this particular way. So this idea of making a larger narrative structure, being able to explain things, that's a non trivial problem. It's a problem. You know, we have the sort of, the very accessible example of LLMs. We have the less accessible example of recording data from human brains and so on. And you know, we have the opportunity with LLMs to sort of to poke them and sort of feed data into them. I did a little bit of that in a piece that I wrote recently about mental imagery of alien minds as called for the internal team. As we were kind of producing this, the, it was called the cat post because it's got a lot of pictures of cats in it. But in any case, the, the thing that is. So, you know, we get to do experiments on alarms that we can't do on brains. You know, in terms of the big picture of predicting human behavior, I think it needs a different kind of formalism. We now have sort of the raw experiments later, or we will have it in the case of brains, we could have it anytime we want it in the case of LLMs. And now the question is, you know, can we make a sort of higher level formalism that tells us things that are of narrative relevance as opposed to just absolutely microscopic information that we can't say much from? I mean, it's kind of like in the case of physics, we can talk about the detailed pattern of the atoms of space, but it tells us nothing about kind of the overall behavior of. I don't know, some gravitational waves coming out from black hole mergers or something that's at many, many levels above kind of the microscopic detail of kind of how the molecules or the atoms of space or something are bouncing around. See? Gosh, many questions here. Let me just see if there's one or two I can cover. Fairly quickly, he says, Lady J asks, can you discuss the history of cellular automata? Oh, my. I was just looking at that for the nth time recently in connection with the historical piece that I'm writing right now. You know, first of all, what are cellular automata? Array of cells. Each cell has a color, say, black or white. There's a rule that says how to update the color of the cell based on the colors of its neighbors. On step before. Okay, that's a very. You know, that idea has been reinvented a zillion times. The earliest inventions, certainly in the 1950s, possibly much earlier than that. I mean, there are examples. One can sort of say, you know, even kind of Pascal's triangle from the 1600s or before is a little bit like that. It's an array of numbers, and you kind of take two numbers and you add them up. If you take Pascal's triangle mod 2, you have a very fine cellular automaton. Rule 90. The. You know, there are things like that. I would say that the kind of one big question is, what was it thought to be good for? And, you know, I reinvented cellular automata in the beginning of the 1980s. I didn't know they had the name cellular automata. And that certainly wasn't their only name. I mean, they were called variously, iterative arrays, tessellation structures, homogeneous structures, automorphisms or automorphisms of the shift, map automorphism of the shift. They had all kinds of names. Nonlinear feedback. Shift registers are very closely related. Many different names for the same concept. Each of those names reveals a different origin story, so to speak. I mean, my choice to call them cellular automata, which maybe wasn't the best thing, kind of sort of reflects back to an origin story that was associated with John von Neumann and Stan Ulam in the early 1950s, where sort of the challenge was, come one, make a kind of a theoretical model of biology. This was before DNA was known. And so on came one sort of idealized biology. And John von Neumann had been trained as a chemical engineer. So his first kind of idealization of a biological system was, let's have it be like a chemical factory. And then I think Oscar Morgenstern convinced him that, no, you didn't really need sort of all of the continuous numbers. It was enough to just have sort of discrete minecraft, like sort of factory construction. And then Stan Ulam, who'd worked on infinite matrices kind of convinced him, no, no, you can just do the two dimensional case. He didn't get to the one dimensional case. Stanulam actually tried to kind of invent a one dimensional case. I don't know why he didn't. I knew Stanulam, but I don't think I ever asked him this question because I don't think I'd read his works carefully enough at the time when I knew him to be able to formulate this question properly. But for some reason he never thought of just the very simple thing that I ended up doing to make one dimensional cellular automaton. He had some elaborate thing with long range connections and so on. I think probably because his intuition was fundamentally if you had something that simple it would never do anything interesting. Which was also von Neumann's intuition and pretty much everybody's intuition, including mine. You know, I was lucky enough to actually sort of use the, the kind of telescope, computational telescope to look into this kind of computational universe of possible programs and was, I would say a decent enough empirical scientist that when I saw something absolutely bizarre that I didn't understand instead of just ignoring it. Well, I admit it took me a couple of years to not ignore it. I eventually realized it was an important thing. But so, you know, part of the origin story was John von Neumann theory of self reproducing automata. The actual name cellular automata was coined by a person called Arthur Burks who I knew, who had been a kind of an assistant of John von Neumann's at the time of the, the Institute for Advanced Study computer project. Von Neumann's computer project. Arthur Burks, in one of those interesting career choices, said von Neumann had offered him a longer term fancier job, I think at the institute. And he's like, no, no, no, I don't want to work on this computer stuff. I want to do something real, I want to work on philosophy. And so he kind of became a philosopher, I mean later on, sort of a philosopher of computation and so on. He had a bunch of students who went on to do interesting things and kind of the intersection between kind of philosophical thinking and computation. But anyway, it's sort of one of those career choices of I don't want to do computers, I want to do philosophy, which certainly is a, is a, well, millennia long. Philosophy is a millennia long story that probably isn't going to get solved at any time soon. But although I think we can have different things to say on the basis of these kind of computational paradigms. But I think what Art Burks was the person who coined the term cellular automaton. I think at the beginning of the 1960s or maybe the late 1950s, again, many places there were many models people have made which were sort of like cellular automata. I've gone on a variety of wild goose chases trying to find where people said, I think somebody told me there was a model like that. Like there was one that I was about to wild goose chase again, a person called Henry Stommel, who was a oceanographer. It was claimed by a person called Ed Fredkin that Henry Stommel had told him that cellular automata had been invented in studying sand dunes in the 1930s. Okay, I haven't managed to find that. I mean, Ed told me a bunch of things that weren't quite sort of true. And this may have been one of them. But in any case, there are a lot of sort of pseudo origin stories like that. You know, the John von Neumann creation of sort of direction of cellular automata and these kinds of realized models of biological systems that became, in the 1950s, late 50s and early 60s, it became a story of can we connect cellular automata as models of computation to Turing machines? Well, yes, endless papers about that. Then this idea of self reproducing automata and the kind of the theory of self reproduction and where would that get used? And I think it was a sign of the times that there was an awful lot of, you know, it was a time when space was really big, early 60s and so on. And it was very much like there were self reproducing lunar factories. Was one kind of idea of take this kind of cellular automaton idea and figure out how to use it to make a kind of Minecraft esque kind of self reproducing lunar factory. So there were branches like that, a quite different branch which is actually closer to one dimensional cellular automata that. Well, one piece of that was nonlinear feedback shift registers. So shift registers are a very natural construct. Once you have electronics and you have kind of a set of sort of electronic elements that you can put in a sequence and then you can have a shift register is you're just taking the value in every element and you're shifting it one place to the left. You can then have a feedback shift register where the whole thing is in a kind of a circle. So when it shifts off the right hand end it reappears at the left hand end. And then you can have more complicated forms of feedback where instead of just taking the one at the end and cycling it back to the beginning, you can take, for example, two or three different taps on the shift register and use some combination of the values at those taps to be the thing that determines what gets shifted back in at the left hand side, for example. And there were studies of nonlinear feedback shift registers. So it's a question of how do you combine those taps? Do you just add the numbers mod2 or do you do something more elaborate? And the more elaborate case, a person called Saul Galam, who I knew quite well, worked on that in the mid-1950s, initially in connection with trying to, well, shift registers. I mean, in the end, linear feedback shift registers, I think might be the most used. The sort of maximal length linear feedback shift register, which is what Sol Ghalam invented, might be the most used algorithm in history. Such shift registers are used all over kind of communication channels, between displays and computers, between GPS satellites for timing information in cellular telephony, for CDMA co division multiplexing. Just a gazillion uses for this algorithm for finding a way of tapping into the shift register so that you will get the maximum length before the sequence of bits repeats. But Sol Gholam also studied the nonlinear case somehow in connection with various kinds of radar timing, kind of things that were kind of analogous to what happens with the GPS satellites today, which is you kind of know what the time is by seeing whether you have a particular piece of sequence that has a particular form. And because the sequence has a maximum length 2 to the n or whatever before it repeats. If you see I have a block of length 15 that is 1, 0, 0, 1, blah, blah, blah, blah, blah, you can figure out there's an algorithm to tell you exactly where in the sequence that block occurs. And so it's a convenient way to get sort of timing information like that. And so that was a use for also was looking at nonlinear shift registers. Didn't, you know, just sort of said, here are some things they do, didn't really conclude much from them. And that was another origin story. Another, another thing was in the 1950s, so long linear feedback shift registers, I should say, were also used in cryptography. It is said that the Soviet military codes were shift register codes. Some people have said, told me that they were linear feedback shift registers that were sort of guaranteed to be of long repeats. And where the inversion of a linear feedback shift register can be done essentially by linear algebra, by essentially matrix inversion. But if it's a big enough register, that matrix inversion is still quite difficult, although with modern computers and modern linear algebra algorithms is pretty pretty straightforward. For the nonlinear case, the cryptanalysis is much harder and is probably NP complete. It's probably a good way of actually making cryptosystems. And for example, my Rule 30 crypto system is a little bit like that. But anyway, it is sometimes said that the Soviet military codes were nonlinear feedback shift registers, although I have to say I've been told more often that they were linear feedback shift registers. But in any case, it was. Nonlinear feedback registers were investigated for cryptography, particularly in the 1950s, and that was done particularly by a bunch of mathematicians in the US who were working for the National Security Agency and who. It's very strange in the literature of this field that there are all these maps of the shift. The shift just means. Well, it's kind of like shift registers, actually, I haven't even ever noticed that. But it's kind of the story of you've got a sequence of ones and zeros and you're kind of progressively shifting them. It's kind of funny, actually, now that I think about it. I'm not sure I ever noticed that before because the concept of the shift map is one in mathematics, kind of. It is really doing the same thing as a shift register in electronics. But anyway, in the late 1950s, a bunch of very distinguished American mathematicians, university mathematicians, were consultants for the NSA and did a bunch of classified work which I think is still not, not, not declassified, but they also published some of their results as these kind of mathematics papers. They're all about the shift map and automorphisms of the shift and so on, and questions like when you have a nonlinear cellular automaton, effectively, under what circumstances is it reversible? Is it the case that every state of the cellular automata and every complete state maps onto a single state so that you can invert it? Well, there was a big table of invertible cellular automata that was published, oh, around 1960, I think, by some of these folks. And I think. Well, they're probably. There are a number of stories related to this which I think are not tellable in a live stream yet. The having to do with the sort of the interaction of sort of the pure mathematics with the sort of military cryptography work. But in any case, the things that. This sort of big table of invertible cellular automata, it was like a thing, and it turns out that was used in the creation of the des, the data encryption Standard, which was sort of the standard form of cryptography which actually has evolved into things like AES and so on, which are standard sort of stream cipher, block cipher, like the standard non public key cryptographic systems that are still used today. But the NDEs for example, it had always been mysterious what the so called S boxes, the things that substitute patterns of bits for other patterns of bits, what had determined what those were. And I haven't really ever tracked down exactly the correspondence between the S boxes and that table. But I'm pretty sure that it was that study of invertible cellular automata that led to the decision to use particular, seemingly random sort of, particular rearrangements of bits in des. So that was another kind of origin story. And I have to say I'm not sure that origin story intersected with any other ones until possibly some talk that I gave at the Institute for Defense Analyses in Princeton. This must have been early 1980s. And I guess the folks that heard about things I was doing with cellular automata and that was, that was a bunch of people who'd worked on kind of practical cryptography and that was a case where sort of these two worlds collided, so to speak. But it was, let's see, I mean other. I would say that the, the vast majority of work that was done on what got called cellular automata was two dimensional cellular automata. A lot of things with image processing in the 1950s and there were a lot of things with sort of so called picture languages was another branch, just many different branches. I have to say that by the late 1970s, many people sort of in the theoretical computer science world had decided everything that could be figured out with cellular automata have been figured out. It's all over now. It was sort of ironic that the kind of experimental investigation of cellular automata that I kind of started at the beginning of the 80s was the thing that sort of was the completely different, very different from sort of the theoretical discussion of that there exists a cellular automaton that emulates a Turing machine doing this or that thing. That had been the sort of the previous focus of work there. That was one focus. I mean another focus of work was we've got this particular way of taking patterns of bits and replacing them by other patterns of bits that is good for sharpening edges and images or something. That was again an example of a cellular automaton. Well, cellular automaton like kind of, kind of method. I mean there were other branches. As I say, it's a concept that is simple enough to, that it's been invented many times separately. I mean there was a whole branch in the Soviet Union, particularly a chap called the lady of. And that was kind of somewhat more biologically inspired I think somewhat in the von Neumann thread. But it led was. I mean there'd been many different branches. So there was a big Soviet piece, there was a big piece in Japan that was particularly concentrating on kind of the somehow the intersection between sort of the signal processing ish version of cellular automata and the theoretical computer science version. There were. Oh, let's see where else I think in the US Mostly cellular automata and research had kind of died out by the time most of the branches had kind of died by the time I started working on it in the early 80s but it was still alive and well in some other places around the world. But that's. Yeah, that's a rough story. I mean there's more detail in my big book New Kind of Science. I have a whole collection notes about the history of cellular automata. Sadly, one fact in one of those notes I believe now to be incorrect as I have determined in the last actually few days and having to do with. Well, it was. I would argue it might not be my fault. I. I spent an hour or two back in the early 1990s kind of interviewing a person who gave me that fact and like are you sure? Was it really this date? Was it et cetera, et cetera, et cetera and. But I got snookered, I think and the, and the actual, the actual date is as many years after the one that I was. Was given. So you know that. But that's to my knowledge that's the only error in the treatment of the history of cellular automata that you'll find a new kind of science. I'll be writing about that one soon. Let's see. Well, I should probably wrap up here. I have to go back to my day job. But a lot of interesting questions here. I see several that prophesied probably do better in my other by. What is it? Is it not called bi weekly? Yeah, bi weekly, yes. But every two weeks iteration of my Q and A about business innovation and managing life. So I think a few of these questions should get moved there. Oh boy. There are so many more here that I would love to talk about and I will. Try and do it another time. I look forward to doing that. Thank you for a lot of interesting questions. All right, should wrap up now, so bye for now.