Future of Science and Technology Q&A (May 24, 2024)

Sam. Hello there. Welcome to another episode of Q and A about future of science and technology. And I see a variety of questions here saved up, I think one that's a follow on from a previous Q and A from Heidi asks, continuing the dinosaur theme which we were talking about a few days ago, is it possible biology can repeat patterns of evolution? Is it possible for dinosaur like creatures to reappear? It's an interesting question. People had asked last time why dinosaurs could appear at the time they did one hundred and something million years ago and the Earth was different. Maybe it had more oxygen. I'm not sure what allows different forms of organism to be produced at a particular time. Some part of it is a historical accident, but it's really a complicated story because I think it's a question of what is being evolved is not just the individual organism, it's the whole ecosystem of what does the big critter eat and how many smaller critters can be supported in this and that way and so on. I think it's very hard to predict. I think it's a typical kind of computational irreducibility. You might know the rules but you can't work out what will happen except by actually running things type of situation. I think there's sort of a question of to what extent the mechanisms that are discovered in biology get rediscovered. Like we know that eyes, for example, were discovered, I don't know, three or four times independently in the kind of evolution of life on Earth. And there are other mechanisms that we see, you know, I don't know how many independent times walking was discovered, but I think flying discovered several times, for example. I think the question of whether sort of a thing that has happened before, like it's good to be really big, like a dinosaur could be a thing that was discovered again or not. And my guess is that it depends on the sort of whole ecosystem of what exists. Whether that can happen. I think it's a hard thing. There's a general question of when can you have a theory for things and when is it just historical accidents? Like for example, when it comes to human history, you can ask how much of human history is the details of some particular leader led some particular country to do some particular thing and how much of it is sort of inexorable patterns of history and people have certainly tried to come up with sort of theories of history of this is what happens when some situation is stable for 500 years or something. Then the bureaucracy starts to build up and eventually it topples over of its own weight or Whatever it is. But there are sort of these patterns of history that. That seem to exist. And for example, if you look at interesting kind of example, if you look at sort of historical maps, and we have these in Wolfram language, we have the data on sort of historical borders of countries throughout recorded history. And if you look, for example, at Europe, where there are lots of countries nearby and where there's a long period of recorded history, you see that. You know, you make kind of a video of the country borders over time. And you see that there are countries that get bigger and bigger and bigger, and then suddenly they. And then they're filled in by lots of other countries. And it looks like a kind of almost biological or physical process of bubbles forming, getting inflated, sort of interacting with other bubbles, reforming, popping, and so on. And it makes one feel that while there might be some aspects, some sort of random seeds that are going on there, that there's some inexorability to the whole pattern of. Of what's going on, even though sort of on the ground, every motion of every border of every country is some complicated war battles, treaties, who knows what. But in the aggregate, it seems to be something about which you might be able to make kind of general laws. And you could say the same thing about biology. There's the question of to what extent is the particular pattern of the feathers of a peacock or something the coincidence of biological history, and to what extent is it inevitable that a thing like that will exist? You know, that raises the question of when you look at sort of forms of organisms, there are certain forms you see and other forms you apparently don't see. And what determines what forms you see, what forms you don't see? I've studied years ago, I studied one particular example of this in some detail, which is mollusk shells, where they form into these sort of tightly coiled pointy shells, and they form into very flat kind of bivalve shells and so on. And you can ask the question sort of what shapes can form? What shapes actually exist that have been selected for in biological evolution? And the surprising thing that I found is that if you have a reasonable model for kind of the growth, the sort of differential growth of different parts of the shell, there's a space of possible parameters. And in the end, every viable space which leaves an actual hole for the animal and things like that inside the shell is actually realized by some biological organism. So in that sense, the overall distribution of shapes of mollusk shells is, in some sense, theoretically predictable. The details of whether this mollusk will live in the Bahamas and this One will live in Chile or something. That is a different story for prediction. But the kind of the spectrum of what possible shapes can occur and do occur seems to be predictable. One comment I made about biological evolution is kind of, who expected the stegosaurus, so to speak, the dinosaur with all those funny plates on its back and so on? Is that something where there's a space of possible dinosaur shapes and that is just one part of the space, and it's sort of inevitable that that something will have sampled that part of the space? Or is that something where that was a particular coincidence of a particular environment, of, you know, some particular forest somewhere, where some particular set of organisms started growing these plates on their back? And that turned out to be a really good idea and there got to be more of them. How does one distinguish those things? Actually, in the recent study that I did about biological evolution and sort of a minimal model for biological evolution, it I'm realizing as I'm talking to you guys that one could actually potentially study some of these questions there of sort of what forms, what's the distribution of forms that occur, and to what extent does one kind of fill the space of possible forms and whatnot, and to what extent can you have a post hoc theory, so to speak, of what's going on? I mean, in my very simple analysis of sort of what can happen in biological evolution, one saying, you mutate these very simple computational rules that are sort of standing in for genomes, and you generate these forms, you generate these complicated patterns of growth. But now the question is, those complicated patterns of growth, they look very different. There are lots of different forms. It doesn't seem like all possible shapes are certainly not being generated there. And the question is, can one somehow characterize what shapes are possible in a kind of post hoc fashion, having sort of an overall theory for that, independent of kind of just knowing the details of what happened from a particular selected computational rule or genome or whatever else. And that kind of potentially might give one some insight into whether there could be a kind of an overall descriptive theory that says, oh, they'll tend to be these kinds of sort of overall patterns of organisms produced versus the only way we can know what's going to happen is by looking at the kind of the specific, by just seeing what the course of biological evolution happened to produce, so to speak, both happened to produce in terms of the coincidences, the accidents of history, and produced in terms of sort of the inexorable. Working out of this rule leads to that thing, leads to this rule, leads to that thing and so on. I think in the kind of when one thinks about that applied to human history, there's certainly science fiction books that famously the Asimov foundation trilogy, in which kind of there's the concept of maybe you could have a theory of history where you could inevitably say this or that thing will happen hundreds of years from now. My kind of concept of computational irreducibility, which by the way, will be 40 years old in a couple of days, that concept is something that really denies that possibility. It really says that a lot of what goes on in the world are things where you can't know what's going to happen until you follow the rules, you see what happened, and you keep going, so to speak. So it really denies the possibility of kind of an overall predictability for history, so to speak. But it doesn't necessarily say that there aren't sort of slices of reducibility in history things you can predict about history, even though you can't say whole story of what's going to happen. And even though as a practical matter, there's sort of a question of to what extent is the course of history something that is determined by the internal dynamics of what's going on, and to what extent is it the chance coincidence of the meteor fell at just the right moment, or whatever else, or the particular sort of person noticed this particular thing that went viral or something at that particular moment? To what extent is the course of history determined by individual random acts, accidents of history? And to what extent is it determined by sort of inexorable processes that can't be predicted in advance, but which are kind of inexorable, given the rules by which sort of the actors of history operate? And those are interesting questions to which I don't think one really knows the answer. But that's some. That kind of, perhaps that gives some context for this very reasonable question of can things like dinosaurs reappear in history? Let's see. Jamie, comments why is it that as technology advances, it goes through phases of bigger to smaller to bigger? I've seen this with phones, computers, TVs, cars. What does that say about the future of technology? I've wondered about this as well, because I always thought it was really nice to have a very small phone, and then people stopped making small phones and they started making big phones or watches, for example, there were very thin, minimal watches, and then there were big chunky watches and so on. And I don't know to what extent that's pure fashion and to what extent there's a little bit of trade off of functionality for size that some things, I mean, sometimes I've thought isn't it just more convenient to put in your pocket a smaller thing? But then it turns out it doesn't make that much difference if it's a bit bigger, but maybe it has more functionality or maybe the screen is bigger. You know, I have a bit, a fairly big cell phone because I like the fact that I can have a big screen. Doesn't matter that much that it's a heavier thing in my pocket. But yeah, it's a, it's a. I've certainly noticed this trend as well that things get bigger and then they get smaller. And like, for example, with computers, you know, I have one sort of now somewhat antique. I think it's called. What is it called? Maybe a Mac book something or other. It's one of the. It's a very small, a Mac laptop that isn't made anymore and you know, it's about the size of an iPad. But it's a, but it's a, a laptop and it's convenient. You can put it in a little bag and it's very light and so on, but apparently nobody cared about it, so it isn't made anymore. And on the other side you can get these, you know, very, very big laptops. Now back in the day, certainly when I first had a portable computer, it was a thing called the Osborne 1 computer and it must have been around, oh, 1980, 1981 or something. It was the size of a small suitcase and didn't run off batteries. You had to plug it in and so on. But it was portable. You could pick it up and then later on. I think the very first portable computer I had must have been 1984 perhaps, was made by a company called Data General that no longer exists. I don't think it was actually made by them. I think it was made by some company maybe in Japan and licensed by them. But it was the DG1 laptop computer and it was, you know, it was a pretty chunky thing, but it had to be pretty chunky because it needed that to be able to have batteries and to be able to have its screen and all those sorts of things. And I think there's a question of sometimes the size of things is determined by things you don't expect. Like in phones, there are antennas that go down the side of the phone and they have to have a certain size just because of the wavelength of the radio waves that they're dealing with. And often in modern times, a lot of the volume of a device, of a portable device is battery. And that is how big it has to be, is dependent somewhat on battery technology. And I guess that there will be sort of waves of improvement or possible smallness as the battery necessary to operate this or that thing gets smaller. Then potentially there's, oh, and the CPU chips get to take less power and so on. I don't know the answer to this question. It's of why things get bigger and smaller. It's a good question. It hasn't really happened, I think, that much in biological evolution, with the notable exception of the dinosaurs, but it certainly has happened in technological evolution. Let's see. Golagaz asks, what will be the future of personal computing, of the personal computing paradigm? Will we see more remote cloud like computing and storage in the future, essentially making personal computing devices obsolete? You know, this is one of these things where it's so cyclic. Is it local, is it remote? And I've seen this multiple times. And back in the day, when workstation computers, back in the beginning of the 1980s, there was sort of the time when a fairly serious computer could be completely on your desktop with its whole powerful CPU chip and so on. And there was that model where it was all on your desktop and had a bitmap display right there on your desktop. That was model number one, model number two. And I knew the people responsible for both of these models, model number two, which was very much sort of liked by the phone company, which was then a very, you know, AT&T, for example, was a very dominant force in the technology world was the idea of having just the local terminal that would have the bitmap locally, but all the computation would be done with connection over a phone line. And so that was a thing that was, you know, there was sort of a business competition between these things, the desktop workstation one, but maybe not for the right reasons, just for reasons of big company, small company, kind of who was entrepreneurially successful, who was not. And then subsequently there were things like X terminals, where the idea was kind of a remote machine and local display and so on. And then when we first built Mathematica, when it came out in 1988, we had this capability to have remote kernels, which we still have today. I use it regularly, where you can have the user interface, the notebook interface, local to your machine. And so you're doing all the typing and editing, all that kind of thing, but when there's an actual computation to be done, it's sent off to a remote machine. Back in 1988, it was a remote machine that could be connected through a phone line, it could be connected through a local area network. In today's world, you would say you send it to the cloud or something, or you send it to some distributed network of computers. I think it's always a trade off between telecommunications speed versus speed of local computers. And that's gone back and forth. And so, for example, right now we're in a moment where the LLMs don't run yet on local computers, they probably will. And that's a place where again, there's sort of a push for centralization, although I don't think that will last. I think the. This, you know, I know for us with Wolfram language, there is an interesting trade off because you can do everything in the cloud. You know, we have cloud notebooks, we've had them for years, and we have had desktop notebooks for what, 36 years now, and you can use either one. The advantage of the cloud is that if there's a big pile of data to download to make something work, it's probably already there and there's no download time. The disadvantage is every time you move that slider to change or manipulate, there has to be a message sent to the cloud machine and back, and eventually it comes back to your web browser and then the graphic changes or whatever. And there's a latency to that process which doesn't exist when you're doing the whole thing locally. But when you're doing things locally, your local machine doesn't have many terabytes of data downloaded to it that might be needed for some particular computation. So there are those trade offs in terms of how that will develop in the future. There's a question of how will telecommunications bandwidth change, how will local storage change? How will you know, is it going to matter that you store bigger things? So I think there are a number of thresholds here. I mean, for example, there was a time when audio got to the point where the things you can generate automatically, you can generate electronically are sort of as good as you can hear with your ears. 44Khz being kind of a pretty good data rate to get audio that is sort of as good as you would hear in the natural world directly with your ears. With video, we're not quite there yet, but we're getting pretty close. And there's a certain rate of sending data which still somewhat strains the telecommunications networks that we have. Not too much, but somewhat. And we have to do quite a bit of compression to have that work. But, you know, there will come A time where there's sort of a limited bandwidth that us humans take in terms of video input. And there are other things where it's sort of a question of, well, what do we actually need, given that the end user is a human, so to speak. And the next big story is things like LLMs, where there's a certain volume of stuff that's needed to produce fluent language or interesting images or whatever else. My guess is that the amount of, sort of the weights and things that exist in current neural nets are very wasteful of the actual amount of sort of information that's needed to drive such a system. And so that will probably come down in size and probably the possibility the sort of the hardware to support LLMs will get more available and so on. And there'll probably be a moment when sort of that big bundle of stuff that is the thing that right now has to be kind of remote, can be made local. Same thing is true of the Web. I mean, the Web, I thought it was an interesting thing. As the web developed, there were times at which local storage was easily big enough. You could just download the web and you could have it locally. Today, I think it's probably the case that you can download the web and you can download a lot of the relevant content on the web and have it locally, but people don't do that. It doesn't happen to be the convention. And people have this idea there's just so much stuff on the Web. Well, there's maybe 10 billion, 5 to 10 billion meaningful web pages, maybe, maybe only 3 billion, depending on what you consider to be meaningful, so to speak. And obviously, the image content, in terms of textual content, it's not that big. And you absolutely can download it and have it on your local computer if you want to. And in terms of images, it's a bit bigger. How much there is in terms of meaningful images. How many billion meaningful images there are on the web, I'm not sure, but there are certainly, you know, that that's again, something where it's not completely out of range to have it be local. And I think the determinant of all this stuff is, well, how much is. It's all determined by what we humans care about. Like, yes, you could download a video in 16K resolution or something, but it's probably higher resolution than our eyes could ever see. Just like you could download audio that contains, you know, things that have hundred kilohertz, you know, 100 kilohertz sounds, but we humans don't hear 100 kilohertz frequencies. And so it doesn't really matter whether you download that or not. So there's kind of a limit to what you need in terms of sort of human accessible kind of data. And I think that somewhat determines whether, you know, at the point when the, when the sort of human accessible data can be local versus when it sort of has to come from some remote source. Then there's the question of at what rate do you need updates about what's happening in the rest of the world? And then sort of the question of how much, what is the bandwidth of the telecommunications capabilities? And the issue is not just the total bandwidth, how many bits can flow through the kind of the electronic pipe, but also the question of latency. After you say, I want that data, how long does it take before you start to get that data? And one thing that is that as 5G rolls out and so on, there's certainly been many claims that, oh, all these things will get better. One of the issues with latency is it's sort of determined by the speed of light. If you want to get something and it's on the other side of the earth, it's just going to take a certain amount of some number of hundreds of milliseconds or whatever, 50 milliseconds, whatever it is, to get the signal, even if it's in an optical fiber going at two thirds of the speed of light or something, because it's not in free space. So it's not going at the speed of light. It's reduced by the refractive index of the glass for the optical fiber. But let's say it's going there. So it still takes some number of tens of milliseconds to just transmit the data. Now you say so that means there's a limit to how low the latency can be. But there isn't really, because if you kind of know what you're going to be asking for, you can cache that data much closer to you. And that's indeed what happens for all sorts of content delivery networks are set up precisely to do that, that they have a whole network of tens of thousands or more of computers distributed around the world, all caching data, so that when people have requested the same thing multiple times, that data can be cached much closer to them. Now, obviously, when there's new fresh data that just arrived somewhere, it has to kind of propagate out from where it arrived to get to the places where it can be distributed to people. But very little of what's going on that's of human Interest is something where you really care. Did you get it in the right 10 milliseconds? You know, it's really, well, you're watching some event or something and you know, if you get the data, if it comes through within a second, you're probably pretty happy. And so I think, you know, that's another determining thing is sort of how do these communications networks work and how much latency does there end up having to be? And that also is affected by things like, well, is it. If you're sending data to some satellite, there's a minimum distance to get out of the atmosphere to get to the satellite. If the satellite is in low Earth orbit, that's kind of modest amount of time. If the satellite is in geostationary orbit, it's just, you know, it's 24,000 miles away or whatever, or more than that. In the slant distance, that's the distance it is above the equator. It can just take a while. But if all you're doing is streaming something from a satellite, doesn't really matter that you get it. That wonderful sports event. What is the moment at which that happens? And what's this kind of simultaneity surface for when people see that if there was a satellite, if there was a delay of half a second from the transmission time from a satellite to one part of the Earth versus another, probably nobody cares. And I think it's. But it sort of depends. There are sort of different technologies. From you've laid fiber optic cables on the ground to you've got high altitude drones that are like 60,000ft off the ground or something that are flying around kind of sort of sending signals and so on to you've got low Earth orbit satellites, to you've got geostationary satellites. And these are things where for drones it's pretty easy to have antennas that are just like the antennas in your handheld cell phone because the distances are comparable to the distances to cell phone towers. For low Earth orbit, it gets a little bigger than that, but it maybe doesn't matter because you just have a large number of kind of local, sort of local access points that are physically bigger objects that are communicating with satellites, but then distributing the data through sort of local radio links that are much lower power and require smaller antennas in terms of. For your actual cell phone. Let's see. So I mean, by the way, in terms of this whole question about personal computing and so on, the other big issue is what are you going to interact with? I mean, this, this, you type and there's a screen, I think has been a pretty successful paradigm. You know, people have said many times that we'll just be talking to our computers. That hasn't worked, at least so far. I mean, there's some tasks, some simple tasks for which it's great to just talk to your phone or whatever, but the more detailed things, the more formalized things. Typing seems to be a pretty good solution, just like kind of when it comes to what we humans do. And human language is our best way of communicating kind of deep, fine detail kinds of things. Visual images are a way of communicating something, too, but they're a little different. And language is the thing that allows us to be the most detailed, the richest, so to speak, in our communication and perhaps the most formal in our communication. And that seems to be important as the big sort of tower of capability that we built from kind of the idea of computation. That's all sort of formalized in that way. And so then the question is, if you're going to communicate in that sort of formalized way, how do you do that? And is it typing or is it something else? Right now, it's firmly typing. Now there's also the question of, well, what screen do you look at? And at this point, there are screens in all sizes, from watches to great big things that fill up walls and so on. And with VR, virtual reality and so on, and augmented reality, there's yet another form factor of screen. And I think for different purposes, these different screens are useful. And I think I've talked before about some of the things that VR and AR will let you do. And is there going to be sort of a computing paradigm that's based on that rather than other things? I think it's sort of interesting to see these things. Like, there are often the new technologies based on metaphors from the old, like the kind of the desktop with all these windows, which are like sheets of paper and so on. And you can imagine what's the 3D version of that for which we don't have a great model, because it's kind of like, well, you know, you could kind of be in a museum and you're seeing these different signs up and so on. Maybe that's a model for sort of the 3D analog. And maybe, you know, the details of history, I don't know. You know, people have had desks and pieces of paper for many hundreds of years now. If you'd gone back to antiquity, for example, I don't know whether the desktop model would have been a familiar one at all, because, you know, paper wasn't A thing. And I'm not sure, although it is amusing that there are pictures on Greek vases and so on that have pictures of people with kind of portable writing desks that look an awful lot like laptops. And, you know, that's a. What determines that form factor. That form factor is not abstractly determined. It's determined by the fact that we humans have a certain shape and functionality. You know, we have those ten fingers, and we're. We're able to type with them. We have, you know, we can put things at a certain. Of a certain size. We can put them on our lap, so to speak. We can see displays of a certain resolution and so on. And yes, in principle, we could have a display that's vastly higher resolution than the thing we have today. But, you know, it doesn't make any difference because our eyes can't see at that resolution. And so those kinds of things, I think are. There's a certain, you know, in the end, it is, how do we fit with how humans work? I mean, you could say, what's the future of chairs? Well, you know, chairs have evolved, office chairs have gotten fancier and so on. But fundamentally, the fundamental operation of a chair is kind of probably the same as it's ever been because we are kind of the same shape as we've ever been. And so I think that's an important determiner of these things. But what means that once you've reached a certain form factor for a device, it's probably once it's been sort of appropriately optimized for the way we humans are, it's probably going to stay that way for a long time just because we're not changing. Let's see. Gosh, gosh. A lot of rather technical questions here. It's too much work for me to do these very technical questions, especially because. Let me see. Let me think. Let's see. SD says current AI and LLM models aren't good at mathematical and statistical computational kinds of things. What areas do you think should be focused on in computer science and mathematics to teach these models to be better at computation and assist researchers and scientists? Well, I just spent the last 40 years trying to build kind of a tower of capability to represent the world computationally and be able to do computation in a way that is sort of suitable to go from how we think about things to how we can compute with things. And I think what we built in terms of that kind of formalized structure is really very good. And when we say, let's take sort of an AI that is A neural net system that's kind of based on what humans can do quickly and easily. It's really different from what this sort of tower of capability that we've built up with sort of formalized computation. And the thing that seems to work well and, you know, has been being done since, oh, I don't know, February of last year or something, and we're about to release a bunch of new things in this area, is you have the LLM as kind of an interface where you can chat with it, so to speak, but then you have the actual computation be done by a computational system, by our computational system, by our computational language, as something that the LLM can communicate with. And so building that interface of having the LLM sort of be able to formalize things as humans might formalize things so that they can be computed with that seems to me to be a very worthwhile area. It's a difficult area because we don't understand that much about how LLMs really work. And so quite a lot of it is kind of hacky engineering, so to speak. Now, engineering in many, many areas can be quite hacky. It's like, well, this happens to work. It's kind of like same thing with, I don't know, surgery in medicine, things like that. It's this thing happens to work, and once it happens to work, let's just keep doing it. Perfectly sensible strategy, but it means you don't get some sort of grand theoretical idea of why things are working. And so, for example, in an LLM, if you say, look, I want a grand theoretical idea of why the LLM responds better to saying please, please than it does to putting the thing in capital letters. Maybe one of those particular things. Just like in artificial neuroscience, you'll be able to put those electrodes into the brain of the LLM, so to speak. You can read out particular activations at particular places in the neural net and say, that's why it responded to please, for example. But to have kind of a global theory of what's going on, I think is difficult. It's not clear. It's possible. It's like having a global theory of neuroscience for brains. It's possible that one can say certain things, but not other things. I think that's an area, sort of the deeper analysis of what's going on and the attempt to make kind of a narrative theory of what's happening in LLMs is a highly interesting thing that probably not as much work is going on at as could be. I'm trying to do some things myself, trying to find sort of minimal models for what's going on in LLMs. That's this recent thing that I did in biological evolution was actually a side effect of that effort of kind of trying to explore sort of how to minimalize kind of adaptive processes of whether evolution or learning. But there's more to do along those lines. But so I think in kind of the. Let's understand what we can understand about alarms. What kind of a theory could we have? You know, I was think I wrote oh gosh, last February or something about how, you know, what is ChatGPT doing and why does it work? Little book that I wrote, I talked there about kind of semantic laws of motion. In other words, if you are generating a series of words and you say the cat sat on the. And you know the next word is going to be mat, can you represent that like sort of motion in a meaning space? Just like we can say if you saw that projectile that was shot in a straight line and had no forces acting on it, then it will continue in a straight line. Newton's first law of motion. The question is, are there similar semantic laws of motion that will allow you to say things about this is how this has to continue in order for it to make sense. Just as you can say the projectile will continue going in a straight line because that's what it has to do in order to follow physics, so to speak. So you can ask similar questions for sort of what's going on in LLMs. I don't know the answer to that yet, but. So those are a few thoughts there. Let's see, Jay is asking, what do you think the amount of data that gets processed or code that is run in terms of bytes versus bytes that are used for storage in the world? In other words, what's the flow of data through computers as opposed to the amount that is stored in the computer? Well, I think there's vastly more that flows through computers, not least because no computer is ever doing nothing. At the very least, it's taking the data from some part of memory and sticking it on the screen. And there's a big flow of, of bits from the computer's memory onto its screen. So there's certainly a lot more. I mean, in terms of. If we say a computer has, let's say, I don't know, for every gigabyte of data that the computer is storing, there's the question of how much is it putting on its screen? Well, let's say it's. I would think. Let me think about this for a second. There's I mean, of course, not taking everything that's in its memory and sticking it on the screen. So let me think. A typical computer might be running at, let's say in round numbers of gigahertz, a billion instructions per second. Those instructions might be operating on, let's say. Let's be somewhat ambitious and say they're operating on a kilobyte of data. It depends. If it's a gpu, it'll be more than that. If it's other things, it might be less than that. But let's say it's that. So that means for every. It's got a gigabyte of data statically, sorry, it's operating at a gigahertz and it's taking a kilobyte of data. So let's imagine that that means it's got 10 to the 12 bytes a terabyte per second of sort of throughput of data. So terabyte per second. And certainly the computer isn't storing the computer's memory. Maybe it's going to be a terabyte. So in some sense the amount of throughput. Now obviously the throughput is measured in different units. It's measured in bytes per second, whereas the static storage is just measured in bytes. I suppose another question one could ask is if we say how long do those bytes last in memory? And most of those bytes will probably be sitting in memory for a long time, hours, days, whatever else, maybe even longer than that. Now we can also ask what about the mass storage? But that's sort of converging. It used to be the case that there was. Well, okay, so in any computer there's a hierarchy of memory from the memory that's right there in the registered inside the CPU that where it's picking up that data and just immediately using it in the instructions that it's running. Then there'll be things like the level one cache, the level two cache. There are different levels in the memory hierarchy of a computer that are typically store progressively more data but are slower to access and slower to get into the actual place where the action is happening in the cpu. And particularly in the past, there used to be much more in the way of sort of mass storage devices. You know, there'd be disks spinning around and so on. Increasingly that's all solid state storage, but still usually slower than the storage that's in the RAM of the computer. But there's sort of a hierarchy of these things. And you can ask the question, you know, I think the calculation we just did of a Terahertz, a terabit terabyte per second going through the CPU of the computer, then maybe your computer might have tens of terabytes of storage that are kind of storage. In its mass storage, so to speak. The next question is, well, how many computers are there in the world? And is all of this processing dominated by little local computers or is it dominated by big server computers somewhere? I'm pretty sure that it's vastly dominated by all the smartphones and computer and laptops and so on as compared to the server infrastructure of the world. And I guess I do know. Let me think. Do I remember? And then am I allowed to say what I know about what the large cloud storage services actually have in terms of, of CPU power? But I think it's. I mean, I think that it's really a question of millions versus billions, so to speak, billions of individual devices versus millions of devices kind of aggregated into kind of cloud systems and so on. So I think. I think the answer is that, okay, so we can also ask a question. What's the lifetime of typical bytes? So how long is it the case that what's the churn time? What's the velocity of data, so to speak? Just like you can talk about the velocity of money and things like that. How many times in a year does sort of money change hands in an economy? Maybe five times, maybe two times. Whatever it is, maybe it's sometimes more than that, but that's a sort of measure of something. You could ask the same thing for bytes in a computer. The typical byte, how long does it take before that byte is replaced? My guess is the answer to that is there'll be some kind of complicated distribution because there's some bytes that last a long time and there's some bytes that are turning over very quickly. And I think it will depend quite a lot on what kind of average you're talking about. Mean versus median versus whatever else. Because I think there are a whole bunch of bytes that turn over very quickly and a long tail of ones that turn over very slowly. So it's a interesting question to try and figure out, but it's one of those estimating questions. I mean, there's questions about, well, how many bytes of storage are there in the world? I know people talk about, what is it zettabytes now of storage in the world. And there's also the question of is there a lot of storage that's on very slow media like magnetic tapes and so on. My guess is that that's actually pretty small compared to the amount of storage that's somewhat online in the computers and phones and so on in the world, I'm not sure, but I think so. Let's see there a question here about from Hybe, what exactly is 5G? How is it different from 4G and 3G and 2G and whatever and 1G? Well, the G stands for generation and sort of. It's partly a marketing story of we've got a new generation of phones. You should go out and buy a new phone. There have been a series of standards and protocols for how cell phones, for example, communicate with cell phone towers. And each new generation has things like, well, we're using different frequencies, we're using different antennas, we're using different encoding of data. They've been getting more and more sophisticated. The typical thing has been, you know, in the very earliest cell phones from the 1980s, it was like, there's a cell, there's a region where there's sort of. Okay, so back in the day, there were radio telephones. Radio telephones were like, you've got your radio telephone in your car and you get a particular frequency allocated permanently for your radio telephone. Then this idea of cellular telephony was that you would get a, you would, there would be a cell, a region where there are a certain number of frequency bands. I think there was 666 was a, was a number I remember from, from the early 80s as a standard from or mid-80s from a standard for cell phones. And then in a particular region you would have that number of channels. And when you sort of open your cell phone, when you start your cell phone, it would, it would try and there would be some hailing frequencies and it would establish a channel to the cell phone tower at a particular frequency. That was kind of the earliest, I think, 1G and sort of 2G kind of approach. By the time it was 3G, I was using code diversion multiplexing. So there were different kinds of ways that you could try and stuff data through these channels. You get allocated different frequency, you get a time slice or you get. So code division multiplexing was a kind of a fancier idea where you use these pseudo random pseudo noise sequences where you have a sequence typically generated by a linear feedback shift register, but it's a sequence of ones and zeros and it's a sequence that has certain properties of randomness. And you take your digital speech signal and you kind of combine it with that random sequence and what you get out looks random. But the point is that each pseudo noise sequence, there are many different Ones, and they're each random in a different way. And so the result is you can take all of this data and you kind of combine it with a pseudo noise sequence. You can kind of combine all those channels together, but you can decode it if, you know, I need pseudo noise sequence number 113, you can sort of take the data that seems like it's a big mess, and you can apply that same pseudo noise sequence again. You can pull out the data that corresponded to channel113 or whatever. And so that was a big innovation in 3G. What was the big innovation in 4G? I'm not sure. But in each different. The general trend has been that is a question of how much data can you really get through this kind of frequency of electromagnetic radiation. We're kind of constrained in what frequencies of electromagnetic radiation we can send through the atmosphere. And then there's a question of, well, how do you put more data through that channel, so to speak. And one of the things that happened is kind of more sophisticated use of, well, you know, this particular pathway had all sorts of interference, and there's different paths from the cell phone to the cell phone tower. And you can kind of disentangle the pieces of the signal that came along different paths and so on. There's a lot of sort of cleverness in the signal processing and the kind of noise reduction that you can do along those lines at 5G. Kind of one of the big ideas is more kind of local repeaters. That is, instead of having something where you are just going your cell phone to the cell phone tower that might be a couple of miles away, it's more like, oh, there's a bunch of local devices. And those local devices might be retransmitting things that they got. Oh, it's an intermediate cell phone, intermediate access point or something, and it's receiving data, and it's then sending that data onto your local cell phone. So the total amount of power that's needed can be much smaller because you don't have to go all the way from the cell phone tower to your cell phone. That's one thing even gets fancier than that. It has phased array capabilities where you have multiple antennas. You have sort of an array of antennas that can direct the radio energy in a particular direction. So a standard antenna, it has a certain antenna pattern where it's sending the radiation, the sending the radio waves in particular directions. So typical, you'd have a dipole antenna which has this particular pattern of kind of these two lobes of energy that it sends out, but by having many little tiny antennas and by arranging them to sort of generate radio signals in a particular phases relative to each other, you can kind of direct the beam, so to speak, to go in a particular direction. And kind of the idea is that in 5G there's notions of, what are they called, pilot waves or oh, maybe it has a different name. But basically there's something which kind of scopes out the local radio environment and then sort of determines how these beams will be sent in different directions so as to optimize kind of how effectively how much the radio energy can be directed in the particular place where you need it, as opposed to just going out into the room at large. But 5G is a pretty complicated thing. And I don't know how long, I don't know how long the standard is thousands of pages. And it'll be one of these things where different people implement different parts of it and it's all a bit complicated. And I think there's also a certain kind of marketing pizzazz of saying we've got 5G and you can be 5G, but you're just dealing with that little tiny piece of the standard that isn't the whole kind of complicated phased array, pilot waves, whatever else thing. And so it's all a little bit unclear, but that's kind of the notion. And the hope is that with all of this kind of local, all the way that this sort of local transmission works and phased arrays and, and differences in the way that packets are sent and so on, the end result will be lower latency, higher bandwidth communication. And there's the potential for things which were previously had to be there. But then there are things where you move your finger and that's controlling something, and it might be controlling something 3,000 miles away. And can it happen in milliseconds, so to speak? Can that control happen in milliseconds? In the end, you're going to be limited by speed of light. But you know, how close can you get to that limit? No good comments relating to, I think some other things we're talking about nowadays. Instead of getting better resolution, we can get higher color range and frame rate. Yeah, well, its color range is great if you're a mantis shrimp with 15 different kinds of color receptors. We only have three. So there's certainly a richness of colors that the actual frequency of light that's generated from a display is very constrained by the fact that it's generated by RG and B, red, green and blue pixels. But I'm not sure that we notice that much. I mean, this is a long running debate of incandescent bulbs versus LED bulbs. LEDs have pretty sharply peaked frequency spectra around red, green and blue that together makes something which is perceptually white to us. An incandescent bulb has this broad spectrum of black body radiation emissions and it has, you know. And can one tell the difference, you know, which one is better than which one? And I'm not sure. I think I'm not as good at telling the difference as some people are. But so I think that, you know, there's some organisms like the mantis shrimp, which has much more, you know, many more color receptors. I don't think we know what it feels like to be a mantis shrimp that has lots of color receptors because, you know, we're used to the colors that we can sense in terms of frame rate. Again, I don't know, you know, at some point our retinas have a certain decay rate and by the time you're at, I don't know, 60, maybe 100 hertz, 100 frames per second, I have to believe you're beyond the rate at which the photoreceptors in our eyes can respond. So, you know, there's a certain sense of, I don't know, a increased perhaps richness and stability of images that we kind of, in some visceral sense. Sense. But I think that when it comes to, I think there's sort of a limit to that. And, you know, it's like different organisms would sense different things. I mean, if you stick. I don't know whether anybody's done the experiment, stick a mantis shrimp in front of a display and see whether it can, you know, what it can tell and what it can't tell. You know, you'd get different results than with a human. I mean, in the case of, you know, cats and dogs, which have two kinds of color receptors rather than three that we humans have, that surely affects, you know, what they get from watching television or whatever. I must say, I haven't seen these movies for cats and dogs. They look, you know, knowing who the real market is, namely the owners buying those movies and so on, videos, it's, you know, they're made for humans as much as they might be made for cats and dogs. Because if they were just made for cats and dogs, they'd have a very different color palette that would look pretty weird to humans. Let's see. Vasco comments. If AI is being used for autonomous vehicles, then presumably technology can improve the level to which vehicles could see around corners by different vehicles communicating with each other and buildings and so on. So the video stream is from multiple perspectives? Yeah, I mean, absolutely, that is a big coming attraction. I know we did some simulations a while back with one of the big car companies about sort of what it's like to have flow of cars around a city and how that would be affected by having both just local sort of autonomous driving activity and kind of globally determined autonomous driving activities. And indeed, one of the things that I think is the most sort of extreme autonomous driving idea is right now we know we drive on the right hand side of the road, the left hand side of the road depending on where you are. But everybody has to do the same thing. But that's a feature of the fact that the cars aren't all communicating with each other. If the cars were all communicating then it will be just fine to say, hey, wait a minute, there's nothing, you know, on this five lane road that really, well, this time of day mostly people just want to go in one, one direction. We'll just have one lane going the other way and we'll, you know, if it turns out there are more cars coming the other way, we'll tell the cars, oh yeah, well now these lanes are closed and you know we're going to have other cars going that direction. You could almost have each individual pair of cars kind of dancing, having this dance on their own as they pass each other. I think there are some parts of the world like, I think there are some islands in the Caribbean where the story is there's an island that was maybe under British control, another one under French control, I remember years ago, I don't know if this is still true, that on the British controlled ones you drive on the left hand side of the road, the French ones on the right hand side of the road. And there are some that are under joint control. And I asked the question, well what, you know, which side of the road do people drive on? And people say, well it's a small island, everybody knows each other and they kind of just negotiate on the spot kind of which, which way they'll pass. And the roads aren't very high speed anyway. But so, you know, one can imagine a version of that for the autonomous cars in the future where it's all done by intercar communication, so to speak. So yes, I think that that's a, I mean in a sense we're beginning to have some of that by all of the sensor technology, you know, all the fact that we're getting data from all these Cell phones that are being carried in all these cars. And so we know, you know, there's a traffic jam starting here. So route cars this way and so on. And that's, that's already to some extent happening, but it's not happening through direct car to car communication. And one assumes that that will be a thing that will happen. How much, you know, how big a deal that will be, I don't know. My guess is it will be, you know, and is it good enough if 5% of cars have car to car communication and the rest are all doing things like having dumb humans actually drive them, or at least not having communications, or not having the same communication standards or whatever else, does it matter that 5% can do this intercar communication if the other 95% can't? It's somewhat related to the whole, the fundamental problem of traffic jams, so to speak. Because one of the weird things about cars is you've got a bunch of cars going down a road and at some point cars are slowing down by five miles an hour, let's say, well, you go back a few miles and that might lead to a traffic jam that has cars stopping and starting. So how can it be the case that when there's just a five mile an hour slowdown somewhere because everybody's looking over at some weird billboard or something like that, as you go further and further back, there's sort of this instability and eventually you have this traffic jam that where people are stopping and starting. Well, it's a complicated thing and there are models of traffic flow, not perfect models, but as you increase the flow rate, all sorts of things happen and you get to these jamming levels and things like this. But in general, the reason why there's this instability has to do with the fact that drivers do not respond instantaneously. And sort of the idea of putting brake lights up higher so you can see through other cars to what's happening further ahead helps, but it's still, drivers have a certain delay time. And that delay time is what produces the instability that causes eventually the stop and go traffic further back in the traffic jam. And it's a. So you know, in the world where every car is an autonomous car and where they're all communicating, maybe there are no traffic jams because maybe there are, because the fact is that there's a flow rate. It's not the case that there needs to be a traffic jam. And it's that yes, the cars may be flowing, there may be lots of cars flowing through, but in the end the cars are going to get through. And it's only because of human delay times and so on that there's the instability that causes a traffic jam to develop. So people wonder, is there a way to have seed cars and things that are sort of preventing those traffic jams? And there are lots of devices that have been invented, like the one car per green going onto a freeway, or all sorts of other kinds of devices, or a big one that was invented, what in the 19th century, 60s I suppose, was the gridlock, avoiding crisscross boxes painted on intersections and so on. Not that everybody always obeys those, but the idea of that was a device for sort of preventing sort of traffic jam buildup and so on. But I think this question of sort of, is there a sort of a trick that you can pull to have, let's say, seed cars driving at a certain speed and holding up traffic and then letting it go at its streaming velocity and so on. Can any of those things help? I think the answer is no. I think fundamentally the problem is human reaction times in driving. And by the time every car is autonomous, that presumably goes away. Whether it goes away when 50% of the cars are autonomous, I'm not sure. And one wonders at that point, will there be lots of autonomous car lanes and so on that operate differently from the other lanes? Things like this. You know, I have to say, to tell a story, a personal story about this kind of thing back in the, oh, gosh, mid-1970s or so, I was. Was curious when I was a kid, I was curious about how traffic jams work and so on. I tried to sort of figure out some math of it. I even actually once spent a little bit of time at some traffic research place where. Where there was a time when sort of a computer, well, not automatically controlled traffic lights were coming in. And it was sort of interesting because there was sort of one group that was the theoretical people who were like, we don't really know why, they had some theory for how the traffic light timings should be based on flow rates of cars from different directions. And so we don't really know that this theory is right. But the empirical people say that it's good. And the empirical people was like, we don't really have measurements that prove anything, but the theoretical people told us that it's a good thing. But any case, I was curious about these things. I tried to make some little models for them and I didn't really get anywhere with it. And then I started working on these cellular automaton systems, very simple programs, particularly ones one dimensional ones where you just have a row of black and white cells and so on. Worked on that for, did a lot of. I've done a lot of things with that. Even have worked on that very recently. This model of biological evolution is actually based on a one dimensional cellular automaton. Okay. But so then many years go by and then I discover one day that the now standard minimal model for traffic flow is a cellular automaton. It's in fact a cellular automaton that I studied. It's Rule 184 in my collection of sort of elementary cellular automaton rules. So I completely missed that. And the model, it's kind of a nice little model that sort of models the basic jamming and unjamming of cars in a one lane highway, so to speak. But it' I happened to invent the underlying framework by which such a model would be made. But when I tried to make a model, my mistake was trying to make sort of too immediately realistic a model of what was happening in the traffic flow. Rather than saying let's sort of zoom out and capture only the very basic essence of what's happening and have that be the thing that's the basis of the model. Let's see. Vaska asks could vehicles on a motorway you reveal your, your, your place is not being in the U.S. the freeways in the U.S. i gradually learned these translations over the decades that I've been in the U.S. let's see. And could vehicles on, on a motorway and those joining seamlessly interweave at high speed safely and traffic be diverted in real time automatically to ensure there are no blockages? Yeah, I suspect the answer is yes. I mean I was just recently happened to be driving in a full self driving Tesla and actually with somebody who happened to have worked on the. On the Tesla autopilot team, his car. And I was a little bit amused that there were moments when he was like I'm about to grab the wheel more or less. But it looked to me like nothing was particularly dramatic was happening. But you know if you know how the sausage is made you get more worried when you see the thing doing certain kinds of things. But it's interesting to see what it managed to do in interweaving in some traffic. And it was a little bit confused but it actually managed to sort it out in the end. Let's see, there's some. Huh. NATO comments. I do notice the red and green and blue when the screen looks pale. Usually the red pigment has gone first. Interesting. You're probably learning something about Your vision, the and no good comments. Turns out that as computer displays get better, our sensor limitations turn out to be higher than we used to think. It's an interesting statement. I mean that we think that the psychophysics of what we can sense is at this level, but once we have displays that are better, we realize that we were missing something. Yeah, I don't know. I mean my own observation is that that, I mean one of the things that's always a mystery to me is that people are prepared to watch a movie on their phone, for example, which just seems very bizarre to me, that people are prepared to, once you're focusing on that thing, that you're kind of prepared to accept sort of a much lower quality, a much lower, you know, smaller number of pixels on your retina and so on. And you know, does it matter that you can see the sort of full peripheral vision that you can get in a nice VR headset or something? Does that matter? Does that really make the whole thing a richer experience? I mean one of the mysteries obviously with videos is that not a mystery really, but just a feature is that the image is one thing, the sound is another. And often one's perception of how high quality the whole presentation is is more dependent on the sound than it is on what you see visually. I should wrap up in a moment here. By store comments. I recall reading saying that 5G and low latency connectivity will be important for self driving cars. That seems somewhat unlikely to be an important component of self driving. Now I agree with you. I remember reading that, I remember those claims and I think people didn't expect that the I'll figure it out for myself on my own as a car would get as good as it's getting. Rab is commenting about the future of fiber optics and that Elon Musk says that SpaceX satellites will render fiber optics useless. Yeah, well, you know, it happens in houses for example. I mean like the house I have that we built, what was it, 24 years ago or something. There's a question of what kind of communication stuff do you run in the walls. And my best guess at future proofing was you run empty conduit. So you can always fish a different kind of cable through those conduits. Those conduits I rarely use. I do have a bunch of fiber optic stuff that was run through those conduits, but the technology hasn't really changed. Fiber optics is fiber optics and those runs at least of a limited size run. There's some things that have happened in terms of the length of the individual fiber runs, but if you're in a house, the length is vastly greater, many tens of kilometers or hundreds of kilometers, vastly more than you need in a house. But it's sort of interesting that yes, all of that run the wires and the walls type thing, a lot of that has given way to just use WI fi. And so the question really I think for terrestrial, for longer range communication is do you run the cables on the ground or do you bounce them off satellites and so on? And for sure it's super expensive to run the cables on the ground. And there's sort of a question of how fast can communications with satellites be and whether one will be able to use infrared and things like this. And that's somewhat determined by the physics of the atmosphere as one uses certain radio frequencies. The atmosphere is transparent. Obviously the atmosphere is transparent to visible light because that's why we get light from the sun. So obviously if you can send things through visible light, which is much higher bandwidth than radio, that's a win. Certainly between satellites there's now laser, I think visible light, maybe infrared laser communication, but not yet, I think to the ground. And so yeah, it seems plausible to me that sort of the WI fi, instead of wires on the walls, analog will play out. Also in the case of more large scale communications. I think it's, you know, it was always an issue when you're sort of putting things on, you know, onto the Internet. What does that actually mean? And you know, where do the wires run? And there are lots of places where wires ran, whether it's the electrical power cables, whether it's along train tracks, things like this, where there was just a place where you could put, you know, a big chunk of new fiber optic cable. And yes, I mean the world will, it will be simpler to do that kind of communication if you don't have to sort of, you know, get out one of those trench making machines and run an actual cable and so on. Rover is asking in the future, will cell towers be more advanced to prevent dead zones? You know, this question about how cell phone towers end up with dead zones, it's kind of a weird thing. I mean sometimes it's because just nobody will let people put a cell phone tower in the place it needs to be put because they just don't want the cell phone tower there and the fake tree isn't good enough for them and so on. Sometimes these things get more Hicksville than you might imagine. I mean, I remember this is now, oh gosh, must be the 1990s. I remember, I was driving in central Illinois and having all kinds of trouble making phone calls and so on. And I bet I could make local calls but not long distance calls and so on. I eventually realized just call somebody and ask them to call the little local phone company and basically say, look, the, you know, this is not working. Your cellular, you know, your cellular, whatever it is, isn't, isn't being able to bridge to the long distance network. And apparently the, the, you know, somebody managed to reach somebody and got on the phone. It's like, oh, whoops, yes, you know, let's reset the switch. Oh yeah, now it'll work. So, you know, I think it's less, less hicksbilly now. It reminds me of a story from this is 19. Oh gosh, 82, 83. I was in England and I had a modem that went over a phone line trying to do computer communication to the U.S. this was a Saturday evening and just couldn't get it to work and eventually call some number. In those days there were more telephone operators. I don't know whether you can even get a human operator anymore on a telephone, but in those days you could. And eventually call and call and eventually we get through to some actual engineer who might have been the main engineer on duty that Sunday evening who says, oh yeah, you dial, you know, this weird code as a rotary dial phone. I think maybe touch tone by that time. No, I think rotary dial still. You dial this weird code and that will switch off this level of echo suppressor and then you should be able to send the data through. And indeed it worked. But those are, you know, the phone systems have become supposedly much better since then. But this question of why there are dead zones, you know, it's often a complicated thing and it's often something where it depends how much foliage is on the tree, how much humidity is there in the air, that will determine kind of the multipath effects of radio transmission and so on. And it can be pretty hard to predict what's going to happen. But yeah, I'm often sort of a bit shocked at places where there are dead zones where I'm really surprised there's a dead zone. And then there are other places where you're out in the middle of nowhere and where cell phone coverage is perfect. Another thing that's a little frustrating is it used to be the case on early cell phones that you could readily bring up some display that would show you the actual sort of cell phone towers you're communicating with and you could pick, I want that tower rather than that other tower. Nowadays that's all been automated and the handoff from one tower to another is automatic. But of course it's tricky because what's happening is the signal strength from the tower you were communicating with is going down. The signal strength from another tower is coming up. When do you hop from one tower to another? And you definitely don't want to be hopping back and forth. You don't want. It's sort of marginal. They're about equal signal strengths. You definitely don't want to be hopping from one to the other and back again. So you kind of have to make a prediction what's going to happen to this signal and so on, and when's the right time to hop. And often it's not really done correctly. And I think that. But it used to be the case that you could like, pick for yourself if you didn't think the cell phone was doing it smartly enough. But. But those capabilities are now long gone on a sat phone, at least the sat phone I have, which is pretty old by now, but still current. The, you know, you can there. It shows you what satellites you're communicating with. And although it's a bit of a cheat because in many places it will just use the terrestrial cell phone network, it only uses the satellites if it really has to, so to speak. All right, well, I should probably wrap up here and thanks for asking about all kinds of interesting things that I haven't thought about in a while and hope you found that interesting and talk to you another time. Bye for now.