History of Science and Technology Q&A (February 8, 2023)

Hello everyone. Welcome to another episode of Q and A about history of science and technology. So I noticed a number of questions which are accumulated here. Let me see what I can do to address them. So IC asks the question, what would you say is the most important human decide that designed algorithm of all time? Interesting question. You know, I think I was curious at one point, what is the most used human designed algorithm ever? And I think the answer is the algorithm that generates the linear feedback shift register sequences designed by a person I knew named Sol Galam. That's a, what that's used for is when you're trying to generate a sequence of bits that you don't want to repeat and you don't want sub blocks of that sequence of bits to repeat. You want it to go for as long as possible without repeating and you want the sub blocks to not repeat for as long as possible. And so what that has the effect of is, is if you take the sequence and you try and say you take one version of the sequence, you take another version of the sequence, they are maximally unlikely to be correlated with each other. And where is that useful? Well, it's useful in many places. So for example, CDMA telephony for cell phones. What tends to be done is that each channel for the cell phone is encoded with a different linear feedback shift or shifted linear feedback shift register sequence. And because these things are maximally don't overlap, they have the best chance to kind of use the same radio frequencies while not interfering with each other. And it's also used in gps. It's the sequences that get generated that give you that maximally let a GPS receiver know what time is it based on the GPS satellite. Where, because you find out where in the sequence are you. The sequence only repeats, let's say every few days and it's running at a very high rate. And you're asking where in the sequence are we? Oh, we're at day three hours, such and such seconds, such and such milliseconds, such and such. And you can tell that because the sequence has these blocks that are sort of maximally non repetitive. It also turns out to be used in all kinds of unexpected places, like what's used in WI Fi. It's used in for the same reason it's used in cell phones. It's used even when you send data from a computer to the display, to the computer display. It's used there because it's convenient in the electronics to never have something which consistently has zero voltage. You always want the thing to be sort of Wiggling around. And this is a convenient way to have it wiggle around and be easy to decode. So I think I worked out how many times in history that algorithm has been used, the algorithm that generates these linear feedback shift register sequences. Very simple algorithm. A shift register is just something where you have a collection of bits and at every step you're shifting one bit off the end and you're having it come back to the beginning again. That's a feedback shift register. Feedback shift registers were first set up in the 1940s initially with, sometimes for electronics, sometimes even they were part of things like delay lines. Like one of the early types of memories was a mercury delay line memory where you would have these ultrasound pulses that would propagate through the mercury, come to the other end, then be amplified, then be sent back again. That was kind of the minimal sort of feedback shift register, which was just trying to preserve its data. When things got more elaborate was when you took a value, you took some of the data and instead of just saying I want to take whatever fell off one end and put it back at the other end, you said, I wanted to have taps in the shift register where I take elements at position 7 and 14 and those are the ones that I take the exclusive or of those things, add them, mod two if they're ones and zeros, and have that be the thing that I put at the beginning of the shift register to be the thing that gets fed back. So that's the kind of idea of a linear feedback shift register. It turns out that some math that comes from theory of finite fields gives you information about what will be the maximal length linear feedback shift register sequences. That's what Sol Galam worked out in the 1950s. And that turns out to be what's used a quadrillion times or something. Maybe, maybe it's more than that. Maybe it's close to an octillion actually times. I worked out that algorithm is the most used algorithm in, in history. I think that is used for, for this thing about transmitting data in such a way that you never have just pure zeros, you always have things that have these, these, these variations. And the variations have these features about being sort of maximally distinguished from each other, being able to know where in the sequence you are, and so on. So that's at least a very well used algorithm. There are algorithms that are also well used in arithmetic. In arithmetic logic units. There are some somewhat non trivial algorithms that don't do kind of schoolbook things like addition, where you're just adding and doing carries it somewhat optimizes how many times you use the data. For example, for multiplication, you would naively think, but sort of standard schoolbook method of doing multiplication of numbers with n digits. Each is every digit has to be sort of compared with every digit in the other number. And you would take N squared operations to do that. But in fact, it's possible to rearrange that computation so that you don't have to do all those kind of combinations. And in fact, one knows one can get N log, log, log or something N number of steps to do multiplication, although the constant in front is quite big. But there are tricks for getting a little bit close to that which get used in pretty much every arithmetic logic unit, every CPU that's built. So again, those are things which have been used many, many times. In terms of sort of algorithms in general, there are several different categories of algorithm. I mean, there are sort of iterative algorithms where you're doing something, you keep on doing it until, for example, something no longer changes. Like the Euclidean algorithm for working out GCDs that is sort of described in Euclid is an algorithm of that type where you keep on subtracting numbers and so on, and until you get down to there being nothing left. And as you watch the progress of what you had to subtract and so on, that gives you the divisors of a number. That's an algorithm which was probably one of the earliest described algorithms from basically described in Euclid. And that again has some significance, although it's not an algorithm that is necessarily used that often. I mean, it's used gazillion times a day, but it's not used at the same kind of high frequency that something like linear feedback shift register sequence algorithm is used. But one general class of algorithms, or the iterative algorithms where you either just do things a certain number of times or you do things until things no longer change. That do things a certain number of times versus do things until things no longer change. The first sort of do things a certain number of times. That's reminiscent of what people call primitive recursion back in the beginning of the 1900s, when people are trying to describe, well, what are the possible operations that you can do that make up something like a mathematical function? And they had the idea of primitive recursion where you just basically say, do this operation. Actually, I should say that a bit differently. I mean, it's something where you can essentially have loops, but you know how many times you're going to have to do the loop General recursion, which actually Godel introduced for Godel's theorem work, is something where you kind of keep going until some criterion is valid. And that's something which exists even in something like the Euclidean algorithm, although it wasn't sort of pulled out in that form. But it's like keep going until something. Until some criterion is satisfied. Another huge class of algorithms are the kind of divide and conquer, truly recursive algorithms, where you take a thing and you break it down and you consider sort of sub cases of that and you build it back up again. Famous example of that is the quicksort algorithm, due to ton from the 1950s, I guess, which was an early computer implemented algorithm. Again, it has this feature that if you want to sort a list, you might think that you have to compare every element with every other and see whether they're in order or not. And that would take N squared operations. But Quicksort manages to do that same kind of sorting in N log n operations by essentially sharing the work in certain ways. So there's a whole class of algorithms that are essentially these divide and conquer, break the problem up into sub cases, do the sub cases, and then bring it back again. There are also lots of algorithms that are kind of pathfinding algorithms where it's like there is a way to get from here to there. There is a way to sort of arrange these tiles so that you can put them in the plane. There is a way to find a series of moves that you can make that get you from one configuration to another, those kinds of algorithms. And there were the Dijkstra algorithm, the A Star algorithm. These were invented, I think, in the 1950s or 1960s that are algorithms for doing this kind of pathfinding. And they get used a lot for lots of practical purposes, I suppose. Other algorithms that are famous are things like the linear programming algorithms, like the simplex algorithms invented BY in the 1940s, I guess, for taking a collection of inequalities and saying what will be the place where these inequalities are all satisfied? Or what will be the place where this quantity is maximized subject to these inequalities? That was the thing that came out of operations research that was developed for logistics in World War II of saying, if there are these constraints between the things that can be produced by factories or whatever else, and you're trying to optimize this particular function, where is the optimum? And it turned out that the answer is that you can describe all of those linear inequalities, for example, as kind of planes that slice or hyperplanes that slice space. And it turns out the answer is always in the corner somewhere. And you have to kind of figure out which corner it is. And it turns out that there's a very systematic algorithm for doing it. It's an algorithm which turns out to, again, it's an exponential algorithm in general. And it then turned out took until the 1980s for people to find sub exponential algorithms for that. And nowadays you can solve problems that aren't just linear slicing problems that are arbitrary convex functions. And that's a whole class of algorithms where a whole class of problems that have become sort of amenable to study because more efficient algorithms are found there. So I suppose one thing to say about algorithms is occasionally there are sort of methods found that let you somehow take a particular thing like pathfinding or like linear programming and suddenly make it more efficient. And then it becomes the case that you go from a situation where you couldn't really use, oh, now you have to do linear programming, or oh, now you have to do pathfinding in the middle of some other problem, to the point where that algorithm has sort of cracked that allowed you to be able to do it. Another big place where that's happened is things to do with neural nets where it was, oh, we want to recognize as this a cat or a dog in this image. That used to be really hard and now it's really easy. And so that's another place where kind of the algorithms that these are sort of, in the case of neural nets, what tends to happen is, you know, you train a neural net and now it will at least approximately implement some particular algorithm. The question of the neural net training algorithm itself, the back propagation algorithm, for example, that's certainly been an algorithm that's been used, that's been applied many times in a typical neural net training these days, it's trillions through quadrillions of those kinds of operations that are being done. You could say that's an important algorithm because it's kind of being used a lot. In that case, my guess is that's actually not the ultimate sort of end point of how neural net training should be done. But that's the way that it's done right now. And that algorithm again uses again another sort of important algorithm in history, which is kind of the steepest descent kind of algorithm where you've got some, you know, you've got some function, you're trying to find its minimum, and you continually try to follow the downward direction as found by calculus by working out the slopes, the downward direction, to get to the minimum. You might not get to the global minimum that way, but you get to some minimum. That type of algorithm, again, very early in history, Newton, I think really did describe what's now called Newton's algorithm for finding, for example, roots of equations or square roots or whatever else where you're successively iterating to get closer and closer to the root. It's again a general method that's been used pretty often. Let's see. Okay, Kat asks here, historically, who has led the trends in science, practitioners or academics? Well, it's varied with the kind of science, it's varied with what you mean by a trend. I mean the general course of the history of science tends to be there is some new methodology invented. That methodology might be a conceptual methodology or it might be an instrumentation methodology. But once that's been invented, there are then lots of things that can be done with it. And usually over a fairly short period of time, five, ten years, something like that, there's sort of lots of low hanging fruit that gets picked. I mean, one saw that early on with a telescope, early on with the microscope. Telescope sort of launched astronomy in a bunch of areas of physics, microscope launched various areas of biology. I suppose in those days the timescale wasn't as short as that. But in more modern times, once one has kind of, I don't know, X rays come on the scene, there are a bunch of obvious things to do with X rays once some methodology, with some physics methodology about, I don't know, working out quantum field theory with asymptotic freedom or something comes on the scene. And what is sort of a set of things to be done there once gauge theories come on the scene in physics, there's a set of things to be done there. It tends to be methodological advance period of sort of highly intense period of kind of picking low hanging fruit, followed by a long cruise period that lasts 50 or 100 years where there's incremental progress before there's some sort of breakout new thing that happens. And depending on what the area is, that incremental progress may be incredibly important in technological and other applications, or it may be something of primarily quotes academic interest that is really being done for the sake of being done, so to speak, which is not a bad thing necessarily, rather than being done for some known external application. So now the question is, if you look at those periods of rapid growth, who does them? Well, in the case of ones that are driven by instrumentation that can be done, I mean, I'm Trying to think in modern times, I think about GPUs as a thing that have driven sort of a bunch of machine learning methodology. And GPUs were very much built by practitioners, so to speak, for purposes that weren't specific to the use case. I mean, you know, telescopes were built by, you know, originally by sort of lens grinding practitioners, so to speak. And the use case that Galileo put telescopes to, you know, look at Jupiter and find its moons, look at the, you know, look at Venus, find its phases, things like this, these are, this was a kind of a use case that Galileo found of something where the core technology had been built by essentially practitioners. I mean, Galileo improved the telescope, but the basic idea of grinding lenses and making a telescope came from, from practitioners of that, of that area. Although those practitioners aren't necessarily the people who make the scientific advance. I mean, I've seen that a bunch of times with, with tools that, that, you know, I've had the opportunity to use where. I'll give you an example. Automated theorem proving is something that has been long studied. There are many practitioners who, who kind of do that. But if you ask what are the things that have been discovered by automated theorem proving that one didn't already know, as opposed to things where you've kind of filled in sort of, yes, we can automate it, automatically prove this thing, just as we could have done it by hand. Well, I think the only example still to date of a surprising result discovered by automated theorem proving was the thing that I discovered in the year 2000 about finding the minimal axiom system for Boolean algebra, which I found by using automated theorem proving tools that have been sort of built by practitioners, but they were not the ones who were thinking about kind of the basic science of what can be done from this. I think we're seeing the same thing again with math formalization tools where people have spent a lot of effort building these. And yes, you can formalize proofs that have sort of already been there. And maybe it's somewhat helpful as a way to organize kind of thinking about these proofs. But if you ask, what are the sort of big leaps of science that you can make using this methodology? You know, I've had the opportunity to do empirical metamathematics where you ask the question, you know, of all these theorems that people are proving a few hundred thousand theorems, what can you say in the bulk about what the structure of mathematics is? That's something, you know, I've been able to do with empirical metamathematics building on essentially Practitioner work that has where sort of the step that is the science ification of that work is a different step from the building of the tooling, so to speak, that will get done by quotes practitioners, so to speak. I mean, one had a very early example of something like this in the Charles Babbage Ada Lovelace story from the 1840s where, where you know, Charles Babbage was very much a I'm going to make mathematical tables, I'm going to build these mechanical computers to do that. And Ada Lovelace, you know, who themed herself more of a sort of a poetess of science, so to speak, was like, well, what's the real overall point of this? And then started realizing things about well, there's this sort of in a sense more abstract idea of computation. And that was a, you know, that was not in the, let me figure out how the cogs can be better made for the machine. That was like, what's the bigger point? What's the ultimate in a sense basic science point there? And so there's some division of labor that can happen in those kinds of things. I think in terms of the question of what counts as a practitioner, what counts as an academic, there's the question of, of when people are primarily employed as academics. You know, I would say that first of all, modern academia is fairly new. Sort of the pattern of the kind of large scale academic enterprise with its very structured organization. It probably dates, well, dates from basically Post World War II and the number of kind of professional academics before that. Well, it was a complicated story, right, because academia was often training for the church, training for the law, things like this, training for these sort of intellectual professions, so to speak. That was what academia was about. Then sort of science got added to academia, mathematics and science got added to academia. And but people who were sort of quotes academics often, you know, there was a, was a complicated thing of were you sort of an academic working in the kind of semi monastic university or were you somebody who was at the court of king so and so or queen so and so and you were sort of the court mathematician or something. And that was not quite the same thing as a university setting. But in a sense you were an academic at some level in the sense that you were paid to just do academic type work or maybe not. Maybe you were sort of that segued into well, you were actually paid to do sort of practical work for, for that particular court. So Leibniz was a good example of that. I mean Leibniz was mostly employed as sort of a, not quite a Diplomat, but a kind of a back room diplomat justifier, so to speak. I mean, a very common thing that he was asked to do was to figure out, you know, so and so wants to marry so and so. But the conventions of the day were that could only happen if so and so came from a bloodline that was, had this characteristic. And that characteristic, okay, Leibniz, go off and figure out, go look at all the sort of historical places and show that in fact it is the case that princess so and so can marry Prince so and so. And you know, he, he did a bunch of study of that type of thing. He also was somebody who generally sort of hung around court and I'm sure he was a fairly witty fellow and people enjoyed that, so to speak, and that's how he supported himself. So was he an academic? Was he a, what was he a different kind of, you know, sort of Court philosopher is a category not so commonly seen in today's world. Although I do feel that there are occasionally there are people who are operating sort of in the mode of court philosophers that you see that even with philosophers, you see that a bit in the kind of Silicon Valley kind of, I don't know what it is, not quite effective altruism. But some of the things that are kind of around the sort of philosophy meets money and maybe some of the things around sort of AI philosophy there you see sort of that type of role that I suppose is the modern version of the court philosopher, so to speak. But I think in when you ask the question, where do significant advances come from? Where they don't come from usually is the vast institutionalized mass of kind of well defined work. That's a rare place for big advances to come from, although there are exceptions. And usually areas of science, once they get past the sort of rapid growth phase and they mature, lots of people start doing them. They get very institutionalized. There's a professional society, there are journals, there are conventions, there's a whole sort of collection of structure which actually makes it much harder for something innovative to happen. Because somebody says I'm doing something different. And people say, well, but if you're doing it, if it's really supposed to be part of this field, it's got to follow these rules and these rules and these rules and well then it can't be that different. So sort of almost by definition the, the sort of major innovative stuff does not come out of kind of the, the, the heavily institutionalized period, you know, areas. I mean, I, I think, you know, people sometimes say, oh gosh, you know, there's no innovation anymore. Look, all these startup companies are all doing the same kinds of things. Well, that's because in a sense, by the time you define your universe of discourse to be startup companies that go through venture capital and do this and do that, well, you've got yourself to a situation that's very much like kind of the normal phase of science where the sort of the highly innovative, we don't really know how this fits together type thing, but there's a sort of big leap that can happen. There has already happened. And you've got to the point where sort of it's a well defined production line and much less sort of dramatic leaps forward can happen. So I think, I mean, in, in, you know, in academia, one of the things that is academia is, is most of academia is well optimized in today's world for this kind of very sort of industrialized areas of, of of science where there's a big group of people doing it and most progress is going to be quite incremental. Occasionally there will be even within that area, for example, somebody who's managed to get themselves in a sufficiently secure position that they can jump out and do something weird that ends up making big progress. Unfortunately, the common sort of sociological point there is by the time somebody has been towing the line for, you know, a decade and a half or something to the point where they, you know, get tenure and so on at their university and all that kind of thing, they've been toeing the line so long that they just like keep toeing the line. And it's people like myself who, you know, I managed to kind of get through the academic sort of system pretty quickly and I was in a pretty sort of secure academic kind of role and I could do kind of things. By the time I was well in my early 20s, I was perfectly, perhaps even earlier than that, I was, I was perfectly in a position to kind of do kind of science that wasn't quite in the main line of any existing area. And it was okay. I could, you know, I could make a living even though I was doing that and that, you know, that was important for me at least in terms of being able to do things that were kind of more kind of different kinds of steps forward than the kind of incremental ones that tend to be done in most of what happens in academia. I think of all these people I know who've discovered all kinds of things, and the story of almost every major discovery is a funky story. It's not, oh, this person was A mainline this, that and the other. They were doing what everybody does in that direction and they just did a little better. And so they got to this new point that's almost never the story. Almost always the story, I would say is it was all a bit unusual. They were, you know, unusually. They'd been trained in this area, but now they were working in this area. They were trained as physicists, but now they're working in biology and they're able to sort of bring one thing to the other. I, I would say maybe, you know, I'm thinking about a bunch of different advances that got made in a bunch of different areas. I mean, Maybe at least 50% come from these kind of outside of the field. You can think of it as people who have sort of funky backgrounds versus the more mainline kind of developments of the field. And as I say, there's often a division between the sort of the tool makers and the users of those tools. I mean, I, I myself happen to have sort of consciously both been in the tool making world and in the using of the tools kind of world. And that's been something I've very intentionally done of. You know, I spend a bunch of time making tools like Wolfram language and then I also spend a bunch of time trying to use those tools. And it turns out often the things that I discover from using those tools actually feed back into the development of new tools. And that's pretty neat when that happens. But it's, it's kind of, it tends to be more that the specialization of your tool builder or you're a user of those tools to do something. And by the way, I mean, it tends to be the case that the kind of, the kind of value system of a well developed area like most areas of academia tends to be one where it's honed in on oh, the, the use of the tools is valuable or the tools are valuable or something else is valuable. And for example, it can be the case that people are doing something that, where they're in an area where it's mostly the write the paper that records the results and so on. And oh, you're building tools and nobody cares about that. You can't, you know, get tenure building tool tools and so on, which is unfortunate because that is actually, you know, some part of where a lot of the value comes is in things like the building of tools. And there can be an area where the sort of, the value system has just kind of excluded those kinds of those kinds of contributions. So that's a, another complicated part of the of that story. I mean, there are many, many specific examples one can give of where, you know, many examples. I don't know why I'm thinking of, I don't know, the discovery of the cosmic microwave background by Penzias and Arno in 1964 or so, where they're working at Bell Labs. I think they had physics backgrounds. I think. Yeah, in fact, that must be the case. Yes. But they were working on satellite communications and they noticed this background radio noise and they knew about the Big Bang theory of the universe and they sort of were able to put two and two together and make a science advance. I think there are. I don't know why. I'm thinking of Kari Mullis, the inventor of pcr, who I don't exactly know. His background is a kind of a wild character, at least when I knew him. But it's kind of a. That was a case where it was sort of a. A more of a practitioner driven. You know, this is a funky, crazy idea. And everybody told him it couldn't possibly work. And he was working for a biotech company and. Well, it turns out it did work and it turns out it's been very important in. Now, is that really, you know, is that sort of a science advance? Is that a tooling advance? It's. PCR has been critical to lots of science advances, but, you know, as a, as a thing itself, it was more perhaps a tooling advance. And yeah, one could go on and on about, about these kinds of things where. So, you know, I think the answer is it's kind of complicated because of this division of labor. What counts as a practitioner, what counts as an academic, all those kinds of things. But in terms of do the big advances come out of the big institutionalized areas? The answer is often not because it's much harder to have a big advance when you're in a very structured institutionalized area. There's a question here. From Philip asking how did Richard Feynman view philosophy? Did you ever discuss non physics subjects with. With Feynman? Absolutely. I discussed many, many, many non physics subjects. You know, Feynman always used to kind of disparage philosophy. One of the amusing sort of footnotes. His son Carl went to MIT and made a point of majoring in philosophy, partly because the old man had so kind of vocally disparaged philosophy. Everybody thought that was pretty amusing. I mean, at least the father and son seem to both think it was pretty amusing. You know, I think Dick Feynman always liked sort of sort of thinking that was grounded and Whenever things started floating off into the. We're just batting words around and we don't really know what the words mean. He didn't like that at all because he really, you know, he was very keen on. I mean, his. One of his greatest skills was. Was calculational ability. And, you know, he used to do this thing which. Which was always amusing, where he would do this big, complicated calculation. And he would think, oh, everybody can do that, which isn't true at all. But then he would go and think, given that he got the answer, he would say, well, what's an easy way that I can derive this without going through all these complicated calculational steps? And he would come up with this kind of great intuitive explanation which wasn't where it originally came from. And then everybody would say, oh, wow. He comes up with these amazing intuitional explanations. His intuition was good, and his intuition about what calculations will work was very good. But it was. It was somehow for him, because sort of the intuitional part maybe was a little bit harder than the calculational part. It was like, that's the thing I should show people. But also, he really liked the idea that you could ground any idea in some very concrete tower of kind of steps. Now, sometimes these intuitional kinds of steps, they sound really good at first, but then you realize, well, wait a minute. That step there. How do you justify that step? Oh, the answer is, it's a big calculation, but we don't necessarily see that calculation. And so it kind of. It's a little bit of a mysterious sort of intuitional kind of thing. But I'm trying to think about discussions about, you know, in terms of. Of people talking about kind of professional philosophy kinds of things. I think he would have been very disparaging of those things. And it's like, you know, and you kind of make fun of these things. You know, people discussing, you know, how many angels hit how many angels fit on the head of a pin. And he'd translate that into something that philosophers might discuss about, I don't know, the philosophy of quantum mechanics, things like that. I mean, you know, it was. Now, in terms of, well, philosophy of life, that's a whole different story. I was. For some reason, I was. Was. You know, Dick Feynman had many kind of philosophy of life points of view. One that I very much ascribe to myself is sort of the. Well, he would characterize it as. There are no experts. I would characterize it more as if you put in the effort, you can understand anything. And I think he was always very negative And I think it's maybe the same kind of thing that I was talking about, about kind of grounding anything you think about in steps you can understand. It was very negative about the situation where the expert will come in and say the answer is X and just trust me, I'm an expert, know the answer is X. He really. I think, as I say, my, my version of it will be more, you know, if you put in the effort, you can kind of, you can understand the steps. I have to say, I'm trying to remember when Dick Feynman was, was, was ill with his. Well, he, he got, was got cancer several times. And I remember talking to him the first time he had that. And he was telling me that the surgeon who was working on his case was writing a paper about his case. And I think he was like, he was kind of, he was slightly amused, but slightly, you know, is that really what you want is to have somebody think your case is so interesting they're going to write a paper about it? But I, I'm, I'm, I'm trying to remember what. Because I certainly talked to him about those. Yeah, yeah, I did talk to him about that. I remember talking to him about, in the case of the, in that medical kind of case about sort of the first principles, understanding of what's going on, which he actually wasn't so much into in that particular case, which was unusual for him. And I think, trying to remember, I, I'm remembering one particular conversation must have been 1981, perhaps 1982. I don't know of that. Around that time in which was, which was kind of a, a come on, understand from first principles this kind of medical thing that was, was relevant to him. But, but I mean, in terms of, of other kind of, You know, he had a, in terms of sort of life philosophy, I think he had this kind of, he had a certain self image about what he was good at, what he wasn't good at, which I don't think was always correct. I think he was sometimes good at things he didn't think he was good at. And, and that, you know, for example, one of the things he, he always used to claim was that, you know, anything administrative, anything that involved sort of people and getting things to happen and so on, he's like, he's not good at that. I don't think that was entirely true. I mean, you know, long before I knew him, when he was working on the Manhattan Project, he was in charge of the human calculators of Manhattan Project. And no doubt he was Maybe he was more of a leader than a. And a strict sort of manager in that area. But that I think seemed to have gone well. But I think later on he was not somebody, you know, there are sometimes academics, scientists who build up kind of large groups around them of people working with them. He never really did that. He was always kind of a solo operative with occasional collaborators who were, I would say, more social collaborators than, than they were sort of strictly scientific collaborators or sometimes collaborators who would, you know, do tool type things of running programs or writing programs or whatever else. But I would say more the criterion for Feynman of collaborators was I would say probably more sort of social than scientific and not the kind of thing where he wanted to build up a big group and he had to manage the group and so on. He didn't want that at all. And he always used to tell me that my sort of interests in managing things and organizing stuff and so on. It's like, you don't want to do that. That's horrible. That's a terrible. I think he never really crossed over the realization that there's, you know, when you can work with a group of people, you can achieve more than you can on your own. I mean, there is always a cost to working with people because you spend a bunch of your effort doing management and leadership and so on. But, but there's also, if you, you know, in the end, there's a lot of leverage to be gained there. I don't think he ever really, really got that. He certainly would always tell me that it was that I was wasting my time in doing these kind of organizational types of things. I mean, I think that for some reason I was thinking about something that he would commonly say which was peace of mind is a prerequisite to creative work. And he always, he took the point of view that it was important to sort of cut yourself off enough from the world that you just didn't have to worry about practical dealing with this or that thing, which would cause, at least for him, some amount of stress. And that sort of disconnecting from those kinds of things was crucial to being able to do creative work. That was something he would say. I didn't always, you know, I didn't really, at the time particularly, I didn't really agree with that. That didn't, that didn't really. But I think it also depends on what aspects of things cause you sort of to not, you know, cause you stress that distracts you from thinking creatively and so on. You know, I think the other thing he would always say, talking. We were talking about academics a little while ago. You know, I have to say that he, he had a definite tendency for being interested in the unusual folk. And, you know, he just wasn't very interested in the whole, well, it's part of the mainline thing and it's part of the, the kind of, you know, we're going to put another, another step in this, in this long chain, so to speak. And he tended to like hanging out with kind of the unusual folk, the artists, the sometimes business people, sometimes the people, you know, with, with different kinds of points of view. I think he always, he always liked that. And he always found that, I think, stimulating. And it was. And so for, for other academics, it was like, you know, that. That almost didn't compute because there tends to be a, you know, academics do tend to, you know, feel most comfortable and hang out most with academics at conferences or whatever else. And I'm, I'm. Well, I could tell you a zillion different, different stories here about Dick Feynman. I'm not sure, um, that, um, um, um, I'm not sure how, how long to go on about that. But, um, any case, Let's see, there's a question here about, from Baker asking if simulation becomes sufficiently good, will it make experimental scientists be out of a job? Well, if we know the fundamental theory of physics and we can work up from that and get to everything in physics, then yes, the experimental scientist would be out of a job because we've reduced physics to a branch of mathematics, so to speak. But there's a big catch, which is this phenomenon of computational irreducibility that just knowing the underlying rules doesn't immediately let you jump ahead and say, so this is how things are going to. This is how everything is going to work. There's this kind of irreducible, computationally irreducible series of steps you need to go through. So just knowing what the structure of the universe is at a scale of 10 to the minus 100 meters or something, then asking, so what does that mean for the biology of bacteria or something like that? You can't get there from here. There's a computationally irreducible sort of irreducible barrier from one part to the other. So in a sense, the existence of computational irreducibility, I think, pretty much tells you that experimental science is not going to be outdone because there are places where you kind of just have to see what the universe does. You can't just go down to the foundations and say, we know what the foundations are. So now let's just build up by simulation. Now, I think it's. And also, whenever you do simulation, that isn't on the bedrock, so to speak, right down at the level of the atoms of space or whatever else, the really the lowest level structure of the universe. Whenever you do that, you're always doing it on the basis of a model. And a model is something which approximates the universe, approximates the world, capturing some features and discarding others. And so if you say, well, on the basis of my model, what's going to happen? Am I going to be able to. I don't know, you know, is this particular kind of process with electrons and this, that and the other, is it going to work this way? Well, actually, electron's a bad example because you're drilling fairly far down there. But let's say you've got some biological thing. Are you going to be able to tell that, oh, I don't know, this electromagnetic wave doesn't affect biological tissue, because, gosh, we can calculate on the basis of a model that this thing will produce this force on this piece of some molecule and this and that and the other. And therefore we know the answer. Well, the problem is you might have missed something. There might be some effect you didn't know. And the only way you can find that out is to do the experiment. So by the time things are turning more into engineering practice and you're asked, well, let's take the following example. Let's take airplanes, and let's take the case of do you use a wind tunnel, as the Wright brothers and others did, to figure out what's the flow of air over this wing and how do I maximize lift, minimize drag, whatever else, how does that work? Well, there was a time when the only thing you could do was use a wind tunnel and just see. But then computational fluid dynamics got better to the point where it was possible to test many kinds of designs using computational fluid dynamics. I think in modern times, there's sort of machine learning that lets you interpolate between the results of computational fluid dynamics and experiments and so on. And you can say, well, you know, if I add this little blob to the wing, is it going to. What kind of effect is it going to have? Well, you can figure that out by simulation. But if you now start saying, well, okay, actually I want to study the, you know, a space plane that's going to, you know, enter the atmosphere@mark20 or something like this, and then you say, well, let me just do a simulation of that Good luck. You know, a lot of. There isn't a. There isn't a sort of. The model isn't well enough set up to be able. Certainly the model of ordinary fluid dynamics won't get you there. But, you know, there are things that are sort of beyond the simple model. And even just doing the computation of the model is really hard in that case. But so I think it is true that once things are developed to the point where you really sort of have them under control, they're part of engineering practice. Then you can probably often just do simulation because you know, the model is good enough, that's going to cover the things you need for this kind of engineering practice. You can just do simulation in the case where things have not been reduced to that. No, you have to do the experiment because what about the effect you didn't think of? So I think experimental scientists won't be out of a job. I think what is true is that there is a transformation of experimental science as a result of sensor data. And the idea, you know, back in the day, it's like we do an experiment and we measure one number and that's what our experiment is about in today's world. It's like, let's actually measure, you know, we're getting a gigabyte a second of data from this experiment and it's a kind of a gusher of data. And both in terms of the getting of the data with sensors and the, well, what do we do with that data? You know, how do we deal with it? Can we just extract one number and say this is the answer? Or are we somehow looking at this data more in the aggregate and so on. And I think there's still lot to be done in understanding sort of how you develop something beyond statistics and beyond machine learning as such, that really deals with sort of the big amounts of data and how you sort of compare theories with experiment and so on when you're dealing with that. I think, you know, statistics was all about, you know, its original idea. It was sort of understanding the properties of a state, like a state in a government kind of state, in terms of a certain set of aggregate numbers. Like what's the overall population? Yes, we can say there's this person with that name and that name and that name that all live in the city. But now we say, what's the aggregate, what's the statistics of this? How many people altogether are there here? How many people altogether died in this year or whatever else? And this sort of reduction of the complete to individual numbers, that's been sort of a story of statistics for a long time. But what we're now getting is a lot of sort of raw experimental data, and it's less clear what to do with that. And it's something where we as humans, you know, we're continually confronted with large amounts of data from our eyes, for example, and, you know, we're getting, you know, megapixels per second and, you know, what do we do with this data? Well, you know, we have all these efforts to, you know, detect features in the data. In the end, the main thing we're going to do is we're going to make some decision to do this or do that, or we're going to sort of make a sort of symbolic output from all of this data. And that's. And, and you know how one does that in science as opposed to just we're making a decision in our life to do this or that. This is, this is a different thing. And I think there's some distance to go in understanding really how does one make use of that large amount of sensor data for science and so on. Let's see, Memes is asking how do we go about solving the goat problem? And I don't know what the goat problem is. Could be anything. I'm sure there's a known thing. I don't know it. I'm afraid David is asking in the history of science, what be the ratio of number of minor paradigm shifts to the number of major paradigm shifts. In this model reminds one of the punctuated equilibrium theory of biological evolution, that things keep going in a fairly steady way and then suddenly there's a big. I think in biological evolution, maybe there are slightly different explanations for that that have to do with the fact that if you're an organism that is mostly walking on two legs and then you gradually make changes to move your center of gravity backwards or whatever else, and then suddenly, gosh, you can stand up on your hind legs or whatever and do a bunch of stuff with your other, with your other legs. And then suddenly it's like, okay, now we're walking on two legs and it looks like a big change, but actually it was more like an incremental change that happened to have essentially a different use case for the same essentially biological structure. I think it's a little different than the case of sort of paradigm shifts in science. I'm sure it's the case, and I don't know how best to measure this, that there are sort of the big earthquakes, the big changes, and then lots of sort of Smaller changes. And I'm sure there's a distribution. Usually these things end up being power laws of the kind of the size of advance, the distribution of sizes of advances where there are big ones that are rare but have a huge effect, and there are smaller ones that are more common and don't have so much effect. And it's an interesting question whether there's a way, using bibliometrics, for example, of looking at citation graphs and things like this. I don't know how long that whole world of citation graphs is going to last, because I think that's sort of part of the academic system that's getting fairly gnarly at this point. In any case, could you make a measurement of this kind of paradigm shift size distribution? I don't know. It seems like a thing where one could at least get some. Some idea of that. Okay, there's a question here from Brady asking what was the fifth class of cellular automata that almost was, which you mentioned in the personal history article that I recently put out? So, okay, here's the story. Back in 1983, I was studying cellular automata, these simple programs where you just have black and white cells, for example, and rules that locally, say, what color to make the next cell. I was looking at these things and I was studied, seen lots of them, and I was like, what do I do now? I'm doing natural science. How do I classify the behavior that I'm seeing? And it was actually rather obvious. I mean, a little bit like, you know, if you say, I've got this material substance and it's a solid, liquid, or a gas, which is it, you know, classify it. There might be corner cases, but basically you can say, it's a solid, it's a liquid, it's a gas. Similarly with cellular automata, it was quite easy to say, class one, the ones where everything just dies out. Class two, it goes to a steady state. Class three, it continues to produce sort of randomness forever. And then, then there was class four, which was sort of intermediate, where it didn't just become constant. It didn't just produce kind of uniform randomness. Instead, it breaks into these localized structures that have complicated interactions. And that's the place where you as a human can most obviously get the sense that it's doing something computationally sophisticated. But I had thought that one could distinguish cases where what would happen is it breaks into localized structures where the structures really are separated from a case where there is sort of a background of sort of crisscrossing type things or something. Like that a background of activity where things don't really separate, but they sort of separate. And I thought you could distinguish those two classes. And that was kind of the Class 5 idea, was to distinguish, I'm not quite sure which order I had them in to distinguish the case where you really are seeing localized structures that separate from the case where you're seeing some sort of significant localized structures, but you're also seeing something that is kind of like a Class 2 system, but it isn't a pure Class 2 system where things just go to constants. Instead they're kind of things continuing to interact in a sort of simple background way, but where the kind of the spice of the whole thing comes from these kind of localized structures and their interactions. So I was kind of distinguishing between the sort of localized structures with no background and localized structures with a background that was interacting with them. But as I looked at that more and more I just found all these kind of corner cases and all these timescales where things would look like they had sort of the crisscross further interaction, but then you run them a bit longer and they kind of simplify out. Or the other way around where it looks like it's quite localized, but then there are things that are more, more global. So I, I kind of gave up on that distinction. And I think, I think it isn't really a hard sort of not solid, liquid gas type distinction. It's more of a thing where there's a range of different possibilities which one can start to study. Question from Universe here has an idea like the Ruliad existed before. Is this a novel object? So the Ruliad is this thing that I started describing a year and a half ago now that is this kind of ultimate limit of all possible sort of ultimate entangled limit of all possible computational processes. So far as I know, it's a really obvious object to study and it's never been studied before. So far as I know. The thing that's tended to happen is people have the idea of let's look for a computer and see what it does. Let's look at possibly something which is non deterministic as many possible computational paths. Let's look at the aggregate of what can happen. Not even the aggregate, whether there exists a path that does this or that thing, but this idea about looking at all possible computations and then looking at the actual computations as they emerge. I think that's just, you know, I, I, I have, I kind of know the people who are sort of still living, who are in this General area, I think, and I think. I think everybody thinks it's kind of a, you know, yes, we should have studied this. I thought, yes, we should have studied this. It's an obvious thing to study, but I don't think it was ever really thought about before. I think what has happened is, and this has happened a lot in our physics project, I've understood slowly that just computation can be thought of as cheap in a sense. There was a time when people imagined that these computational steps, they're very expensive. There's just one path. We just look at that and so on. But the idea that the universe can just be an absolute unbelievable waster of computation, the idea that there can be all these different cases that are all just running in parallel and everything's happening, and most what we sense is just this unbelievably tiny slice of all the things that are happening. The idea that there is computation out there, there's huge amounts of computation going on out there, and it just happens, and it's not expensive, so to speak. That's an idea that I think, for example, for me, I've only. It's taken me a long time to sort of get used to that idea. And for example, the notion that there might just be a tree of all possible kind of evolutions for the universe and even just the equivalencing of different sort of different states of the universe which are the same state, happens only through the sort of the perception of the observer, so to speak. And that really what's going on is this unbelievably huge treeing out of possibilities that that is a reasonable point of view and not just a how can you think about that? It just involves too much. Computation is a fairly. Something that's only slowly emerged. I think one thing really helped me is in the 1980s, started thinking about kind of the physical world as being something that was. Could be described computationally and that really processes in nature are just computations. The. The things we see in nature are a mere physicalization of what is ultimately a computational process. And when you start thinking about computation as the stuff that happens in nature, the idea that there can be just lots and lots and lots of it becomes much clearer. And so I think that's kind of the path through which one gets to that. Let's see. Twister is commenting. Neural nets that combine seemingly unrelated fields of research can produce great results, they say, but our academic and business cultures are more focused on specialization. What are my thoughts on that? Well, okay, first point. It is the case that there is an increasing realization that neural nets can be generic. You can have a neural net that's useful for language generation, it's useful for protein folding, it's useful for some image processing, image kinds of things. There's sort of a generic thing a neural net does. I make two comments about that. You could say, well, gosh, golly, isn't it surprising that this one computer can do all these different kinds of things? But that's the idea of universal computation. That's the idea. You can have that single Turing machine that is universal that can do all these different kinds of computations. That's one point. That to say we've got the system that is sort of can do all these different things is perhaps less surprising than it might be. Now having said that, neural nets are not Turing machines. Neural nets are one as they're usually deployed, are kind of one shot kinds of things. You just have this big network of neurons and so on and the data comes in at the top and it percolates down and you're computing from one neuron to the next, one layer to the next, and then you get an answer at the end. And that, that kind of sort of single path computation, it prevents the possibility of having arbitrary length computational steps. It really, it crushes out any computational irreducibility. Now the fact that there are individual neural nets that go across a lot of different areas, my guess is the reason that happens is in many cases is because they're doing things a bit like what humans do. And the things that we have chosen to identify as areas we care about, whether it's language generation or image construction or whatever else else, are ones that sort of resonate with the way that we humans do things. And so that's a. So in a sense it's kind of the one neural net, just like we have the one brain that is sort of this, this thing that operates in a particular way. That's sort of why we see the sort of the, the same neural net showing up usefully in different places. Because those things are all, are all sort of made use of by the same, by a brain with the same structure, so to speak. Now the question of whether sort of the cross fertilization of different areas is important to progress in academic and business and so on. Absolutely. That's almost without exception. The really energetic stuff is happening at the boundaries between existing areas where you apply an idea from this area to that area and so on. And people have said, I don't know all the time I've been around at least they talk about interdisciplinary research and they're very concerned about how do you take these silos of research, for example, and have something at the boundary? I would say that that's gone up and down in popularity. I remember when I was first involved in doing science sort of professionally in the late 1970s, it was the idea of an interdisciplinary conference was a wow, this is a new thing. This is kind of like the physicists can be at the. The same conference as the biologists. Wow, that's never happened before. But the truth is that in the actual doing of those fields, a lot of what had happened was, you know, physicists had gone into biology, discovered all kinds of interesting things and so on. So, you know, that kind of cross fertilization had certainly happened. I think that, yes, the challenge is when things get big, which includes science, they're hard to organize. The best way to organize things is to modularize them, typically. And when you modularize, you create silos. And that's why it's hard to have these things that sort of go. When you have a big operation going, it's hard to get it to deal with the things that cross over between these different areas. I mean, I know in. Even in. In my day job of. Of, you know, CEOing a tech company, you know, in fact, just today, I spent an hour and a half grinding on with all kinds of wild, you know, historical analogues, grinding on this issue of we've got these different areas. We're working on this thing and this thing and this thing and this thing. And now we've got things we need to do which are new and different, never been done before. That involves sort of crossing over between this area and this area. And it's like, but who's going to do that? We have people working in this silo. This silo, this silo. It's challenging to put those things together. And that's often where sort of leadership is important, is just defining. We are going to have this new thing. It will not emerge from silo A or silo B. It's got to be something that somebody, some crazy person has to come and say, we're going to. We're going to invent this thing, and it's going to take things from silo A and silo B, but it isn't in either of those silos. And so it's a challenging thing to do even when you're dealing with a company that has definite leadership and so on to. To even, like, take those ships and steer them in the right direction. Is quite, quite difficult. And it's something that is when, when you're dealing with more of the kind of what's out there in the world and the institutional structures that are out there, it's not going to spontaneously happen. It's going to spontaneously. The structures that exist are going to replicate themselves and the. Let's make this bridge to this other structure. It's very unlikely to happen. And so, you know, is it the case that, that we'll have sort of systematic ways to take, you know, ideas from place A and place B, merge them together, make a new idea in case C? There are certainly amusing historical examples. There's this thing, it was called triz, which was a Soviet era concept in the, in very much the, you know, the planned economy, planned science kind of Soviet point of view, there was this kind of idea that, well, we could, we can plan everything, let's plan innovation. And so the concept was you kind of take an idea from one area, idea from another area, you kind of cross them over and that will generate deterministically a new innovative idea. I don't think it works particularly well. There are, you know, there are examples you can give where it's worked. But you know, in, in general, that is a thing that is, you know, you can, you can imagine, you can get a little bit too structured about, structured about how to make unstructured innovations. You know, I will say that in my own life and times, knowing a bunch of different areas and being able to make analogies as well as bring in tools from different areas has been fairly critical to a lot of things I've done. And certainly with our physics project and the things that have emerged from that, these sort of wild analogies between, you know, I don't know, the structure of space, time and theories of gravity and things that you can say in metamathematical space about kind of the future of mathematics and so on, these kind of wild analogies have a lot of power when they work out. Because what will often happen is that in one area, the sort of, that the branch of the analogies in that area will have been well worked out for 100 years. A lot will be known. But then in the other area it's very fresh, it hasn't been worked out, but you can now import the things from the well developed area into the less developed area and oh gosh, you can then take the less developed area and suddenly sort of physicalize it, bring it up to a level of development, let's say, that's comparable to what's happened in physics, for example, it's something I was doing recently with metamathematics, physicalization of metamathematics, of sort of taking ideas from physics and importing them into the much less well developed area of metamathematics. I should probably wrap up really soon here, but I see a question from Rudo here about what would a modern analog computer look like. Well, first of all, let's talk about analog computers. I don't know when the first things that we would call analog computers really existed. The first sort of structured. I mean, we could make up a lot of things as being like analog computers. Like slide rules, for example, from the 1600s, you know, slide rules for those who aren't as ancient as I am and use them at one time. You know, slide rules were used until the early 1970s when calculators kind of took over. But a slide rule is this thing where you have this scale. It's usually a logarithmic scale. And you have this sliding part and you have this cursor that reads between the scales and, and you use the fact that if you literally move a slide out in some direction and the slide is marked in logarithmic increments, then when you look at, you know, the fixed piece and the slide that was moved out a certain direction distance, you can multiply numbers just by looking at sort of how the, the, the, the logarithmic units on the slide sort of add to the logarithmic units on the, on the main part of the slide rule and so on. So you can think about that as a bit of an analog computer. And those existed certainly from the 1600s and so on. I would say that the more developed analog computers. Oh, I don't know. Kelvin, for example, in the 1870s, 1880s, developed an analog computer for working out the tides, where he would have. Essentially it's like a clockwork machine because the tides have these different components. I think he was doing with 10 components and he was doing something where you're kind of rotating one wheel and that makes another wheel go this amount. You're adding up the rotations of all these wheels to compute the tides. I suppose you could say it's a little bit like the Ptolemaic system in astronomy. And presumably the Antikythera Device from around 0ad is this computer that has, has gearing. And I suppose one could think about it as sort of an analog computer too. Where, where the analog, where the gears that can be moved to any, the, the wheels can be moved to any angle. It's not digitalized, although it looks a bit digital because it has, you know, it has teeth on the gears that are used to mesh the gears together. And I suppose also are used to be to have the feature that, you know, if you have two gears next to each other, the, the ratio of the number of teeth will be the ratio of the, of the rates of rotation of the different gears. But I think that that's. But I think, you know, I've used analog computers a few times back in the day. I remember using one at Los Alamos National Lab back in the beginning of the 1980s. And in those days, so those analog computers were electronic and they had. Like, you know, any old passive electronic circuit, you know, LRC circuit, inductor, resistor, capacitor circuit, there's a differential equation associated with that circuit. And you know, if you want to solve the equation for a ball bouncing on a surface or something like that, you can actually. That's a little bit weird because it has a weird condition to it. But let's say you want to solve, I don't know, a spring bouncing up and down, you want to solve that differential equation. You can convert that differential equation to an equation to something where you just have characteristics in an electric circuit. Then you can just see what the time behavior of the electric circuit is and it will give you what those equations were like. So in a sense, what you're doing is you're saying, I've got these differential equations, for example, and I've got this electric circuit which satisfies the differential equation. Can I map the equation that I want to the equations of the, of the electric circuit? Now, an electric circuit like that is fundamentally linear in its operation. So that means that if you've got a heavily nonlinear system, it's hard to map it to that linear differential equation that can be implemented just using kind of standard electronics and so on. I think. Well, an interesting question and people certainly tried this is, you know, can neural nets be made analog? I don't think it actually matters that there are all those digits of precision in the neural nets. So it's kind of like, well, if you were worrying about making 32 bit floating point numbers, then yes, it might be worthwhile to have them in analog circuitry. If it turns out it's four bit numbers, it's less important to be an analog circuitry. I think people regularly say, let's use an analog computer for this. Let's use it for some kind of image processing. Let's use something where we're doing Fourier transformation forms by doing some diffraction type thing, sort of just using physics. I mean, by. In a sense, when you say an analog computer, what you're saying is we're not going to go through trying hard to make the idealization that data is represented in bits. We're just going to have the data represented by, I don't know, an electromagnetic field representing some light at this place in a hologram or something like this, and not say, specifically, I want to bitify the thing and then see what happens to those bits. I mean, even, you know, any computer, one's taking those bits and one's doing sort of quotes, analog physics to go from one configuration of bits to another. But you're concentrating on, oh, and the thing, the state of the system is represented in terms of bits. An analog computer, I think sort of the defining feature is you never represented in terms of the state of bits. It's just the state of some system that could include something you imagine is a continuous variable. I think, well, people, you know, there are photonic quantum quotes, quantum computing companies that are very much doing sort of the optical equivalent of analog computing. So I think that that's kind of what one would expect the future of these to be is probably, particularly in the optical domain, using sort of the properties of a wave front of light or something like this to represent the data, rather than saying, oh, it's made up in terms of these bits. That's kind of the way one would expect that to go. I mean, mostly analog computers didn't really make it in the. Probably in the 80s and so on, because there was sort of a huge. The programmability of analog computers is very limited. I mean, when I remember when I was using, you know, it had plug boards and you could connect this to that. And no doubt, I think it even had some ability to have an ordinary digital computer specify where the, you know, where the switches will be on and off to determine which part of the system would be being in use and therefore which analog function was being run. And I suppose one could imagine a situation where it's all digital up to the point where it's specified what the circuit should be and then it runs an analog. But I think at that point you're getting so close to just having a pure digital computer with maybe one additional component. And then you say, well, I'm going to read it out digitally anyway. So then why does it matter that there's this analog piece in the middle? That's, I think, the reason why we're not seeing so many analog computers in the world. All right. I should probably wrap up there. So thanks for joining me, and thanks for lots of interesting questions, and I look forward to doing this another time. So thanks and bye for now.